 Fri Feb 28 17:33:16 2020, Urs Langenegger, Full test, Fulltests on 2020/02/28
|
M1589 B
M1590 C (gain issues?)
M1591 C (noise issues?)
M1592 B |
|
Thu Feb 27 10:14:09 2020, danek kotlinski, Module transfer, M1562 to DESY
|
M1562 was not qualified during a full test since the test did not complete.
Looking at the module I see that roc 2 has a lot of noise row 79.
I tried to mask all pixels in this row but the masking does not work.
With this feature one can run pixel alive but trimming fails completely for roc 2.
The only way to trim the module is to disable the whole roc 2.
Since it seems to me that we will never want to use this module in the detector
I have given it to Jory to be used in the DESY test beam.
She already has 2-3 module from the pre-production, so with older versions of HDIs & ROCs.
I have also looked at 2 other modules which were classified as C 1557 and 1558.
Both look fine to me but I have not run the PH optimization on them.
D. |
|
Thu Feb 27 09:47:51 2020, Urs Langenegger, Full test, Fulltest on 2020/02/26
|
On Wednesday, 2020/02/26 the following modules went through the full qualification procedure:
M1581
M1582
M1583
M1584
All received grade B (from a cursory glance due to mean noise). |
|
Wed Feb 26 10:34:55 2020, Urs Langenegger, Full test, Fulltest after Peltier replacement
|
On 2020/02/25, after SIlvan had replaced the Peltiers, I did a fulltest. Here the summary:
Finished all tests. Summary of test durations:
Fulltest@-20 127 min 5 sec
IV_TB0@-20 4 min 23 sec
IV_TB1@-20 5 min 1 sec
IV_TB2@-20 5 min 1 sec
IV_TB3@-20 5 min 1 sec
Cycle 39 min 23 sec
Fulltest@-20 121 min 6 sec
IV_TB0@-20 4 min 23 sec
IV_TB1@-20 5 min 1 sec
IV_TB2@-20 5 min 1 sec
IV_TB3@-20 5 min 1 sec
Fulltest@10 115 min 17 sec
IV_TB0@10 4 min 42 sec
IV_TB1@10 5 min 5 sec
IV_TB2@10 5 min 5 sec
IV_TB3@10 5 min 5 sec
--------------------------------------------------
total 461 min 48 sec
All modules were graded 'B' (I think because of the noise mean being to high in a few chips per module) |
|
Tue Feb 25 17:11:18 2020, Urs Langenegger, Module assembly, Modules glued and tested Feb 24/25
|
Modules glued and tested Feb 24/25
The letters after the module name indicate the reception test grade.
M1590 A (after Silvan fixed broken HV bond)
M1592 A
M1595 B
M1596 A
M1597 A
M1599 C (leakage current: 16uA)
M1600 A
M1598 had a broken clock wire bond, was diagnosed by Wolfram and fixed by Silvan. Will be processed together with the other modules that had problems after taking them out of the storage box (M1593 and M1594), in case they can be fixed. |
|
Mon Feb 24 14:01:29 2020, Urs Langenegger, Module assembly, Cap gluing in W8
|
In week 8 the following modules were glued and tested:
M1581
M1582
M1583
M1584
M1585
M1586
M1587
M1588
M1589
M1591
M1590 was not glued last week because it had one HV bond broken (Silvan fixed this on Monday, 20/02/24).
M1586 initially had problems with the readout after gluing, but this was fixed by porperly closing the MOLEX connector. In the reception test, M1586 was graded 'B'.
All other modules were graded 'A' in the reception test.
Overall the double-gluing setup works very well. The modules above were glued basically in one day. |
|
Mon Feb 24 13:53:32 2020, Urs Langenegger, Module assembly, Setup changes in week 8
|
Setup changes in week 8
Silvan considered the single-module gluing approach suboptimal. It was changed:
- we switched sides, the cap gluing is now done on the window-side of the aisle
- there are now two jigs set up for cap gluing
The procedure itself for gluing did not change: Apply the stamp with the glue to module 1, release, put module 1 into its jig, insert module 2 into gluing jig, apply stamp (because of the glue fluidity, the marks from module 1 will have disappeared and there is no need to re-apply any glue to the stamp). The caps are lowered onto the modules only once the glue has been applied to both modules.
Silvan recommended applying the weight of only one copper(?) bar, to be placed centered on the top of the z-stage.
Silvan also recommended against using ethanol. Instead we are supposed to use aceton. As a result, the plastic tweezers should not be used anymore, but rather metallic tweezers.
Finally, Silvan changed the Peltiers, and applied isolation inside Red October. We observed cooling times from 10C to -20C of 6 minutes. |
|
Fri Feb 14 14:40:59 2020, Dinko Ferencek, Reception test, RT for 2 modules: 1578, 1580
|
1579: Grade A
1580: Grade A |
|
Fri Feb 14 14:05:16 2020, Dinko Ferencek, Module assembly, Production yield so far
|
Of 41 modules produced and tested so far (1536-1576), 6 modules were found to be bad before or during the reception test, 5 were graded C after the full qualification (one of which is possibly C*) and the remaining 30 modules were graded B (of which 4 were manually regraded from C to B).
The overall yield for good modules (A+B/Total) produced so far is (30/41)=73% |
|
Fri Feb 14 09:48:27 2020, Dinko Ferencek, Full test, FT for 12 modules: 1561-1576 (1563, 1567, 1572, 1575 excluded)
|
Feb. 11
1561: C/B/B (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits in ROC 6, but trimmed Vcal threshold distribution and distribution of trim bits look OK. IV: 0.26 uA at +10 C, slope 1.07
==> Manually regraded to B
1562: TEST INCOMPLETE: pXar crashed during 2nd Fulltest@-20 (m20_2). For more details, see here.
1564: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.78 uA at +10 C, slope 2.25
Feb. 12
1562: C/C/C (-20/-20/+10). TEST INCOMPLETE: SCurves test failed for all ROCs in all 3 Fulltests with the following error messageERROR: <PixTestScurves.cc/scurves:L277> no scurve result histograms received?! Trimming does not work in ROC 2.
1565: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.39 uA at +10 C, slope 1.07
1566: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.37 uA at +10 C, slope 1.07
1568: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.13 uA at +10 C, slope 1.15
1569: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.16 uA at +10 C, slope 1.09
1570: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.18 uA at +10 C, slope 1.08
1571: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.43 uA at +10 C, slope 1.09
Feb. 13
1573: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.25 uA at +10 C, slope 1.09
1574: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs, problems with bump bonding and trimming in ROCs 1 and 2. IV: 0.46 uA at +10 C, slope 2.14
1576: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.23 uA at +10 C, slope 1.09 |
|
Thu Feb 13 21:44:28 2020, Dinko Ferencek, Reception test, RT for 6 modules: 1573, 1574, 1575, 1576, 1577, 1578
|
1573: Grade A
1574: Grade B, I(150)/I(100) > 2 (4.63)
1575: Grade C, problem with one TBM core
1576: Grade A
1577: Grade A
1578: Grade A |
|
Wed Feb 12 15:40:13 2020, Dinko Ferencek, Reception test, RT for 3 modules: 1569, 1570, 1571; 1572 bad
|
1569: Grade A
1570: Grade A
1571: Grade B, I > 2 uA (3.09 uA)
1572 from the same batch of 4 assembled modules was not programmable and was not run through the reception test. |
Wed Feb 12 11:15:04 2020, Dinko Ferencek, Module grading, Problem with dead trimbits understood 14x
|
It was noticed that some modules are graded C because of typically just one ROC having a large number of dead trimbits. One example is ROC 10 in M1537
At the same time, for this same ROC the Vcal Threshold Trimmed distribution looks fine
as well as the distribution of trim bits
It turns out that the trim bit test is failing in this and other similar cases because of the tornado plot that is shifted up more than is typical
In the trim bit test code, the Vthrcomp was raised from 35 to 50 but for cases like this one, this is still not high enough. We therefore further increased the value of Vthrcomp to 70.
Grading procedure:
We will manually regrade to B all those modules graded as C due to dead trimbits provided the Vcal Threshold Trimmed distribution looks fine, the number of trimming problems is below the threshold for grade C (167), the distribution of trim bits looks reasonable, and they are not graded C for any other reason.
Side remark:
It should be noted that the Trim Bit Test distribution and the way the number of dead trimbits is counted will not always catch cases when the trim bit test algorithm failed. For instance, in the MoReWeb output for ROC 10 in M1537 one does not immediately see that there are many underflow entries
which arise from the fact that the untrimmed threshold used in the test is too might (Vthrcomp value too low) and Vcal that passes the threshold is not found and left at the default value of 0
This problem of not spotting the algorithmic failure is particularly severe in the case of ROC 2 of the same module M1537 where it goes completely unnoticed in the summary table
it is very hard to spot from the plot (because there is no statistics box showing the underflow)
but the problem is there for a large number of pixels
Even from the tornado plot one would not expect problems
but this tornado is for one particular pixel (column 12, row 22) and there is no guarantee that for other pixels the trim bit test won't fail. |
|
Wed Feb 12 10:43:04 2020, Dinko Ferencek, Module transfer, 9 modules sent to ETH: 1539, 1545, 1547, 1548, 1549, 1550, 1551, 1552, 1553
|
Yesterday the following modules were sent to ETH:
1539, 1545, 1547, 1548, 1549, 1550, 1551, 1552, 1553
The FullQualification tar files for these modules have been copied to the common CERNBox folder shared with ETH and the transfer dates have been added to the assembly spreadsheet. |
|
Tue Feb 11 17:56:06 2020, Dinko Ferencek, Cold box tests, Fulltest failure - psiAgente
|
During today's full qualification the 2nd fulltest@-20 for TB1 (M1562) failed
psiAgente: self.Testboards[1] failed- the following boards are busy: TB0(DTB_WRBSJ9:M1561), TB2(DTB_WRE1O5:M1564)
This looks similar to the report made on Jan. 30.
Looking at the pXar terminal window I managed to catch the following output
Thread 1 (Thread 0x7fe20f554b00 (LWP 7987)):
#0 0x00007fe20c4e90cb in __GI___waitpid (pid=17338, stat_loc=stat_loc
entry=0x7ffe23b71480, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1 0x00007fe20c461fbb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
#2 0x00007fe20e998172 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x1fa2570) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:2119
#3 TUnixSystem::StackTrace (this=0x1fa2570) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:2413
#4 0x00007fe20e99aa43 in TUnixSystem::DispatchSignals (this=0x1fa2570, sig=kSigSegmentationViolation) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:3644
#5 <signal handler called>
#6 0x00007fe20d5d6794 in PixTestTrim::trimTest (this=this
entry=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:359
#7 0x00007fe20d5da500 in PixTestTrim::doTest (this=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:132
#8 0x000000000040a3d0 in main (argc=<optimized out>, argv=0x7ffe23b7b828) at /home/l_tester/L1_SW/pxar/main/pXar.cc:376
===========================================================
The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6 0x00007fe20d5d6794 in PixTestTrim::trimTest (this=this
entry=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:359
#7 0x00007fe20d5da500 in PixTestTrim::doTest (this=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:132
#8 0x000000000040a3d0 in main (argc=<optimized out>, argv=0x7ffe23b7b828) at /home/l_tester/L1_SW/pxar/main/pXar.cc:376
===========================================================
However, looking at the relevant section of the code in tests/PixTestTrim.cc
356 for (unsigned int iroc = 0; iroc < rocIds.size(); ++iroc) {
357 for (int ix = 0; ix < 52; ++ix) {
358 for (int iy = 0; iy < 80; ++iy) {
359 if (thr2[iroc]->GetBinContent(1+ix,1+iy) > initialCorrectionThresholdMax - 2) {
360 thr2[iroc]->SetBinContent(1+ix,1+iy,0);
361 }
362 }
363 }
364 }
there is nothing special there, just getting the histogram bin content. Could this be some random memory corruption issue? |
|
Tue Feb 11 10:25:32 2020, Dinko Ferencek, Reception test, RT for 6 modules: 1563, 1564, 1565, 1566, 1567, 1568
|
Feb. 10
1563: Grade C, ROCs 8 and 10 not programmable, no obvious problems with wire bonds
1564: Grade B, I > 2 uA (2.17 uA)
Feb. 11
1565: Grade A
1566: Grade A
1567: Grade C, no data from ROCs 8-11, looks like a problem with one TBM1 core
1568: Grade A |
|
Sun Feb 9 19:34:55 2020, Dinko Ferencek, Full test, FT for 4 modules: 1557, 1558, 1559, 1560
|
1557: C/C/C (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits, failed trimming and mean noise >300e in ROC 4. IV: 0.20 uA at +10 C, slope 1.08
1558: C/C/C (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits in ROCs 2 and 12, mean noise just above 300e for ROC 2, possibly C* module. IV: 0.27 uA at +10 C, slope 1.07
1559: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.23 uA at +10 C, slope 1.09
1560: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.24 uA at +10 C, slope 1.09 |
|
Sun Feb 9 18:36:13 2020, Dinko Ferencek, Module grading, Manual grading
|
The procedure for manual grading is described in https://github.com/psi46/MoReWeb/pull/120.
In short, inside the test folder it is necessary to place a text file called grade.txt that contains just one number representing the manual grade:
1=A
2=B
3=C
Addendum:
It appears that manual grading does not work properly if one reruns MoReWeb with grade.txt added. It is necessary to first delete any entries related to the FullQualification using the following command
python Controller -d
after which you need to specify for which module you want to delete entries, e.g. M1537. Once done, you need to run MoReWeb for this module, e.g.
python Controller -m M1537 |
|
Sun Feb 9 12:05:47 2020, Dinko Ferencek, Module assembly, Cap gluing
|
| Today protective caps have been glued to modules 1561 and 1562. |
|
Sun Feb 9 10:52:17 2020, Dinko Ferencek, Full test, FT for 4 modules: 1539, 1554, 1555, 1556
|
1539: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.17 uA at +10 C, slope 1.08
1554: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.44 uA at +10 C, slope 1.80
1555: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.99 uA at +10 C, slope 1.30
1556: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.87 uA at +10 C, slope 1.67 |
|