ELOG Layer 1 Replacement

HDIs: 3001, 3002, 3003, 3004 and 4033

HDI test

Today I tested 5 HDIs: 3001, 3002, 3003, 3004 and 4033
Results:
3001
- electrically OK
- HV OK
- I heard 10+ sparks during 60sec
--> HDI did not pass tests
Danek took it for HV test at his setup

3002-3004, 4033
- all showed different patterns of electrical test failures in Quadrant 0 and 3 (Q0, Q3)
- either all 3 (clock, CTR, SDA) tests fails on Q0 and/or Q3 or some of them or only Ch1 (out of two channels) fails.
--> HDIs did not pass tests
Since patterns were similar the reason could be miss-alignment of contacts. Under microscope in some cases one could see marks outside contact pads.
Conclusion:
we have to re-align the pin head of the jig with respect to HDI and repeat tests of HDIs: 3002-3004 and 4033.

101

Wed Mar 11 17:12:05 2020

HDI testing procedure change

HDI test

There is an additional test that will be used from 11.03. on all HDIs. It involves measuring the voltage between ground and the HV pin of the needle card with the goal of checking whether proper voltage is delivered to the HDI. The HDI testing script has been updated and it now prompts the user to set the voltage to -800 V, measure the voltage on the pin and write this into the test results. After this, another instruction has been added which tells the user to raise the Z-stage (needle card) before setting voltage to -1100 V. This is done to prevent sparking from the HV pad to the HV pin if the alignment is slightly off.

While measuring the voltage on the HV pin, one should keep in mind the proper settings on the Keithley. When the voltmeter probe is connected to the pin, the current reading on Keithley will go up and if the current readout range is low, it will limit the voltage even before hitting compliance. This was observed during the initial testing of the new procedure. Range should be set to maximum (100 muA) and compliance should be set to 105 muA. This is high enough that it is not reached while measuring voltage at -800 V.

Wed Sep 18 15:45:45 2019

HDI sparking under HV test

HDI 8010 passed all test except the HV. Under the HV test, some sparks could be heard and even seen by eye on the right TBM (closer to the HV pad!). Testing it again, showed that it was indeed damaged by the spark. Sparking craters could be seen with the microscope between pads 4 and 5 of the TBM.

The same thing happened a month later, on HDI 8012. Again, it passed all the test until HV when sparks appeared and damaged the TBM, and, curiously enough, the damage was on the same position on the TBM as with 8010.

HDI 8011 was put under the HV test as well, however, this one didn't have any wires bonded to the TBMs because gold plating was missing on several of the wire bonds. Sparking could be heard, but we couldn't located where the damage happened. There is one potential candidate, shown on the photo attached to this log entry.

On 18.09. a possible explanation for the sparking was thought of. Wolfram and Matej put one of the "faulty" HDIs under HV test and noticed that sometimes a spark could be seen between the HDI handle and the aluminum base plate! It looks like the HV pin rests near the border of the HV pad and the ending of the HDI. Because of that, there is a discharge from the HV pin to the HDI handle. The discharge connects to the ROC wire bond pads closest to the HV pad providing a way to affect the TBM! This is further strengthened by the fact that the damaged TBM pad is connected to the first four ROCs closest to the HV pad. To test this, we put kapton tape on the HDI handle, near the HV pad and tested the HDI again and there were no sparks.

The HDIs to be used for L1 replacement are slightly longer to prevent HV jumping to the sensor. Incidentally, this might also solve the sparking issue. If the issue is not solved on the longer HDIs, HDI holders will have to be further isolated in the region close to the HV pad.

Attachment 1: HDI_8010.jpg

Attachment 2: HDI_8010_unzoom.jpg

Attachment 3: HDI_8012.jpg

Attachment 4: HDI_8012_2.jpg

Attachment 5: HDI_8011_1.jpg

Attachment 6: HDI_8011_2.jpg

Attachment 7: HDI_8011_3.jpg

Wed Aug 7 11:49:07 2019

HDI glue irradiation tests

Six different glues were used to glue the cap on six dummy modules. They were all irradiated to 120 MRad in Zagreb after which they were taken to PSI and tested with the tweezers under the microscope.

Standard glue (Dow-cornig), used in CMS - it became "two-component". Solid part on the capacitors and the liquid part on the cap. Surface tension was actually holding the cap quite strongly. Silvan deemed it the second best glue

Two component epoxy adhesive - the strongest glue out of all tested. The only drawback of it was that during the application of the glue to the capacitors, it left puddles around some of them since it is quite liquid before curing. This might have been prevented if we had a proper glue stamp. The glue stamp used at the time contained too much excess glue, and if another one was printed, with small grooves where the capacitors are, the amount of glue transferred should be lower and no puddles should appear. This glue was graded the best of all the tested glues. EDIT: the glue stamp with smaller grooves was printed and now there are no puddles anymore.

SG-20 (black) - sticks, but not really well. It was also quite soft and malleable. Because it doesn't stick very well, it is not recommended for gluing the cap.

WS-200 - almost identical to the SG-20, just a different color.

Terosan MS939 - doesn't stick, not recommended.

Ergo 6521 - doesn't stick, not recommended.

Took photos of them under the microscope. There are two photos for each glue. First one is while the cap is at rest, while on the second there is an upwards force applied with the tweezers. All of them except the two component epoxy adhesive detached when force was applied. With the epoxy adehsive, the module started lifting, but the glue held.

Attachment 1: dowCornig_con.jpg

Attachment 2: dowCornig_split.jpg

Attachment 3: epoxy_2component_con.jpg

Attachment 4: epoxy_2component_split.jpg

Attachment 5: sg20_con.jpg

Attachment 6: sg20_split.jpg

Attachment 7: terosan_ms939_con.jpg

Attachment 8: terosan_ms939_split.jpg

Attachment 9: ws200_con.jpg

Attachment 10: ws200_split.jpg

Thu Feb 6 17:09:50 2020

Gluing stamp improvements

Silvan has made additional grooves on the gluing stamp which now sits stably on top of the module without any rocking motion and also allows better glue distribution to all the HDI components that are supposed to receive the glue.

107

Sat Mar 14 17:08:14 2020

Fulltests on 2020/03/13

Modules tested:
M1591 C (gain at -20)
M1595 B
M1597 B
M1598 B

Mon Mar 2 17:39:36 2020

Fulltests on 2020/03/02

Modules tested:
M1595 C
M1596 B
M1597 C
M1600 C

Andrey and I suspect that these grades are driven by a bad PH optimization.

Fri Feb 28 17:33:16 2020

Fulltests on 2020/02/28

M1589 B
M1590 C (gain issues?)
M1591 C (noise issues?)
M1592 B

Thu Feb 27 09:47:51 2020

Fulltest on 2020/02/26

On Wednesday, 2020/02/26 the following modules went through the full qualification procedure:
M1581
M1582
M1583
M1584

All received grade B (from a cursory glance due to mean noise).

Thu Jan 30 13:51:03 2020

Fulltest failure - psiAgente

Cold box tests

During the first fulltest@-20 of the full qualification procedure for modules: 1539, 1541, 1542 and 1543; an error message popped up "psi Agente: self.Testboards[0] failed - the following boards are busy: TB1(DTB...:M1541), TB2(DTB...:M1542), TB3(DTB...:M1543). This means that the first fulltest for module 1539 was not executed.

Second fulltest seems to be working fine (so far)

Cause of this problem is not clear.

Tue Feb 11 17:56:06 2020

Fulltest failure - psiAgente

Cold box tests

During today's full qualification the 2nd fulltest@-20 for TB1 (M1562) failed

psiAgente:  self.Testboards[1] failed- the following boards are busy: TB0(DTB_WRBSJ9:M1561), TB2(DTB_WRE1O5:M1564)

This looks similar to the report made on Jan. 30.

Looking at the pXar terminal window I managed to catch the following output

Thread 1 (Thread 0x7fe20f554b00 (LWP 7987)):
#0  0x00007fe20c4e90cb in __GI___waitpid (pid=17338, stat_loc=stat_loc
entry=0x7ffe23b71480, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007fe20c461fbb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
#2  0x00007fe20e998172 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x1fa2570) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:2119
#3  TUnixSystem::StackTrace (this=0x1fa2570) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:2413
#4  0x00007fe20e99aa43 in TUnixSystem::DispatchSignals (this=0x1fa2570, sig=kSigSegmentationViolation) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:3644
#5  <signal handler called>
#6  0x00007fe20d5d6794 in PixTestTrim::trimTest (this=this
entry=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:359
#7  0x00007fe20d5da500 in PixTestTrim::doTest (this=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:132
#8  0x000000000040a3d0 in main (argc=<optimized out>, argv=0x7ffe23b7b828) at /home/l_tester/L1_SW/pxar/main/pXar.cc:376
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6  0x00007fe20d5d6794 in PixTestTrim::trimTest (this=this
entry=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:359
#7  0x00007fe20d5da500 in PixTestTrim::doTest (this=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:132
#8  0x000000000040a3d0 in main (argc=<optimized out>, argv=0x7ffe23b7b828) at /home/l_tester/L1_SW/pxar/main/pXar.cc:376
===========================================================

However, looking at the relevant section of the code in tests/PixTestTrim.cc

356  for (unsigned int iroc = 0; iroc < rocIds.size(); ++iroc) {
357    for (int ix = 0; ix < 52; ++ix) {
358      for (int iy = 0; iy < 80; ++iy) {
359        if (thr2[iroc]->GetBinContent(1+ix,1+iy) > initialCorrectionThresholdMax - 2) {
360          thr2[iroc]->SetBinContent(1+ix,1+iy,0);
361        }
362      }
363    }
364  }

there is nothing special there, just getting the histogram bin content. Could this be some random memory corruption issue?

Wed Feb 26 10:34:55 2020

Fulltest after Peltier replacement

On 2020/02/25, after SIlvan had replaced the Peltiers, I did a fulltest. Here the summary:

Finished all tests. Summary of test durations:
Fulltest@-20 127 min 5 sec
IV_TB0@-20 4 min 23 sec
IV_TB1@-20 5 min 1 sec
IV_TB2@-20 5 min 1 sec
IV_TB3@-20 5 min 1 sec
Cycle 39 min 23 sec
Fulltest@-20 121 min 6 sec
IV_TB0@-20 4 min 23 sec
IV_TB1@-20 5 min 1 sec
IV_TB2@-20 5 min 1 sec
IV_TB3@-20 5 min 1 sec
Fulltest@10 115 min 17 sec
IV_TB0@10 4 min 42 sec
IV_TB1@10 5 min 5 sec
IV_TB2@10 5 min 5 sec
IV_TB3@10 5 min 5 sec
--------------------------------------------------
total 461 min 48 sec

All modules were graded 'B' (I think because of the noise mean being to high in a few chips per module)

261

Mon May 11 21:37:43 2020

Fixed the BB defects plots in the production overview page

Software

0407e04c: attempting to fix the BB defects plots in the production overview page (seems mostly related to the 17 to 10 C change)
f2d554c5: it appears that BB2 defect maps were not processed correctly

260

Mon May 11 21:32:20 2020

Fixed double-counting of pixel defects in the production overview page

Software

As a follow-up to this elog, double-counting of pixel defects in the production overview page was fixed in 3a2c6772.

265

Wed May 13 23:16:37 2020

Fixed double-counting of pixel defects in the production overview page

Software

Dinko Ferencek wrote:

As a follow-up to this elog, double-counting of pixel defects in the production overview page was fixed in 3a2c6772.

A few extra adjustments were made in:

38eaa5d6: also removed double-counting of pixel defects in module maps in the production overview page
51aadbd7: adjusted the trimmed threshold selection to the L1 replacement conditions

Wed Oct 2 12:41:16 2019

First production modules assembled

First production modules (M1530, M1531, M1532) were built on Monday, Sep. 30 2019.

There was a problem with wire bonding of ROC5 on M1530. There is a small crater on one of the ROC pads which appears to had been created by the BareModule probe card needle.

After initial tests with pXar of the 3 modules, noticed the following:

M1530: data missing from ROCs 4 and 5 (expected based on the wire bond problem on ROC5) but otherwise looks fine
M1531: could set Vana but setvthrcompcaldel and pixelalive not showing any data
M1532: looks fine.

Caps were glued to M1530 and M1532 (damaged cap was glued to M1530) and Reception tests were run before and after gluing. Attached are the bump bonding threshold maps before and after cap gluing and the (before - after) difference for M1530 and M1532.

Attachment 1: M1530_beforeGluing.png

Attachment 2: M1530_afterGluing.png

Attachment 3: M1530_diff.png

Attachment 4: M1532_beforeGluing.png

Attachment 5: M1532_afterGluing.png

Attachment 6: M1532_diff.png

Mon Jan 27 17:50:48 2020

Cold box tests

Failing modules at -20

Friday, 24.1., it was noticed that the modules are graded as 'C' in similar fashion. Module is graded A/B at +10 degrees and then graded C at -20. The reason for grade C was that the trimming procedure was not successful on at least one of the ROCs on each module. This problem was further investigated on 27.1. Wolfram suspected that the problem lies in zero suppression mode since the pixels with failed threshold comes in groups of four pixels. The corresponding DAC 'ctrlreg' was set, as recommended, to 17, during previous tests and after changing it to 9, the problems at -20 seem to be gone. This will be further investigated.

211

Tue Apr 14 17:19:44 2020

FT of M1668, M1669, M1670, M1672

M1668: Grade B due to mean noise >200e for several ROCs
M1669: Grade B due to ROC2 mean noise >200e
M1670: Grade B due to ROC1 mean noise >200e
M1672: Grade B due to mean noise >200e for several ROCs

218

Thu Apr 16 17:31:16 2020

FT of M1662, M1675, M1676

M1662: Grade C due to failure of ROC4 in almost all tests: PixelAlive, PH calibration etc
Should be investigated and retested. At Reception PixelAlive etc was OK, only one double column showed problems
M1675: Grade B due to mean noise > 200e for several ROCs
M1676: Grade B due to mean noise > 200e for several ROCs

206

Thu Apr 9 17:24:49 2020