ELOG Layer 1 Replacement

M1562 to DESY

M1562 was not qualified during a full test since the test did not complete.
Looking at the module I see that roc 2 has a lot of noise row 79.
I tried to mask all pixels in this row but the masking does not work.
With this feature one can run pixel alive but trimming fails completely for roc 2.
The only way to trim the module is to disable the whole roc 2.

Since it seems to me that we will never want to use this module in the detector
I have given it to Jory to be used in the DESY test beam.
She already has 2-3 module from the pre-production, so with older versions of HDIs & ROCs.

I have also looked at 2 other modules which were classified as C 1557 and 1558.
Both look fine to me but I have not run the PH optimization on them.

D.

Wed Mar 4 11:32:42 2020

change the target trim threshodl to vcal=50

After some discussion we decided to change the target trimming threshold from vcal 40 to 50.
Many ROCs cannot be run with xrays at 40 while all I have seen until now can be run at 50.
45 might be possible for some rocs but will fail for others.
D.

135

Wed Mar 25 14:42:55 2020

modules 1545 & 1542 back from ETH

M1545 & M1542 were returned from ETH to PSI for further testing.

141

Wed Mar 25 20:17:16 2020

The list of modules tested today by Wolfram

Module doctor

M1542 nothing on sdata3 (12-15), bad output or wire-bond
M1544 ROC15 timing off, can be made to work (roc 15 vdig 11, ..), but probably not usable for the detector
M1545 full readout, ok, upgraded to C*
M1546 TBM1 output alpha broken (sdata1) => replace TBM1
M1563 ROC 8 thinks he is roc 10 ==> check address wire bond on roc 8?
M1567 roc 11 missing, sdata2 (b-channel)
M1572 sticker says : both TBMs broken, no SDA => replace both TBMs?
M1593 tbm0, core B (roc 0-3), no decodable output, core/stack working ok, output beta+ high ~ 2V
M1594 roc 0 dead
M1615 nothing on sdata 3 (12-15), very bad cable (corrosion?) re-running with new cable
M1616 ROC 10 not programmable, no roc headers on sdata2 (b) ==> TODO check clock to roc 10
M1617 readout but no hits in ROC 8, mechanical damage on chip edge
M1621 ROC 8 not programmable, no roc headers or tbm trailers on sdata2a (tbm1b) ==> TODO check clock on ROC 8
M1623 probably no cal-trig-reset to roc 0-3
M1625 definitely not cal-trig-reset to roc 0-3 (CTR- at 0V), damaged when trying to pull wire-bonds
M1575 no readout from rocs 0,1 (tbm0b, sdata4), all rocs programmable ==> TODO check CTR roc 0

142

Thu Mar 26 10:41:24 2020

M1606

I have looked more closely at M1606.
Trimming show problems in ROC2, 8 pixels have 0 threshold and for 61 pixels thr=-1.
Strange because from Scurves the 61 have a high ~62 threshold but no failure.
The PixelAlive and PH is OK for these 61 pixels, the 8 are just completely dead.
Trimbit test shows that 61 pixels failing.
So maybe this is the first time we see a real trimbit failure!

I suggest to grate this module C+( or *).

155

Sun Mar 29 18:14:59 2020

Modules 1544 & 1563 in gelpack

The 2 bad modules:
M1544
M1563
Have been moved to gelpacks.

161

Tue Mar 31 06:54:12 2020

M1542

I have tested M1542.
It looks not bad but there is a group of pixels in the lower/left corner of ROC6 which behave different.
One can also see that something is not right in some PH optimisation plots.
See the attached plot.

We should probably leave this module as grade C+.
D.

202

Thu Apr 9 14:19:40 2020

Modules moved to gel-pack

The following good module class A/B have been moved to gel-packs:
1607, 1608, 1609, 1610, 1612, 1613, 1614, 1618, 1619, 1620, 1622, 1624, 1626, 1627, 1628.

The following bad module, not-working and C class have been moved to gel-packs:
1544, 1563, 1567, 1675, 1593, 1594, 1611, 1615, 1616, 1617, 1621, 1625, 1646, 1650, 1652

205

Thu Apr 9 16:13:51 2020

Modules moved to gel-pack

danek kotlinski wrote:

A few corrections to the list of bad modules:
- On April 7 M1593 has been (F)-tested with a grade B due to Rel.gain width and mean noise of a few ROCs.
- 1611 should be C* since ROC13 has not working double column (160pixels) + 27 trimbit failures but very good trimmed threshold distribution, so these 27 trimbit failures should be ignored.

213

Wed Apr 15 08:48:09 2020

Wolfram's tests from 14/4/20

Module doctor

two of are fixed and can be re-tested
M1623 tbm bond
M1657 bond roc 15

the others need further investigation, maybe a new tbm, maybe closer wire-bond inspection

M1633 roc 0-3 programmable, but no readout, unclear
M1635 roc 12-15 programmable, but no readout (except for roc 12), no token passed
M1653 roc 12-15 not programmable, otherwise ok, sda? tbm?
M1671 some problem with roc 14/15, unclear

214

Wed Apr 15 17:14:42 2020

move 4 modules to gel-packs

Moved to gel-apcks:
1629 B
1631 B
1660 C
1665 classifed as B in MoreWeb but has 170 pixel failures

216

Wed Apr 15 17:33:53 2020

move 4 modules to gel-packs

danek kotlinski wrote:

Moved to gel-apcks:
1629 B
1631 B
1660 C
1665 classifed as B in MoreWeb but has 170 pixel failures

M1665 is graded B since there is no a single ROC with >4% of damaged pixels (max 120 in ROC5). 170 pixel failures are totally in the module.

242

Thu Apr 30 15:38:43 2020

M1635 & M1671 transferred to gel-pack

Two bad modules have been placed in gel-packs: 1635 & 1671.

246

Fri May 1 19:34:01 2020

M1582

M1582 was classified as C because of 167 pixels failing trimming in ROC1.
I have tested this module.
The attached plots show the 1d & 2d threshold distributions.
The average threshold is 49.98 with rms=1.39 there is 1 pixel failing (at 0) and 1 pixel with a very low threshold of 37.
I think this ROC is OK, actually it is very nice.
D.

258

Mon May 11 14:14:05 2020

M1606

On Friday I have tested M1606 at room temperature in the red cold box.
Previously it was reported that trimming does not work for ROC2.

In this test trimming was fine, only 11 pixels failed it.
See the attached 1D and 2D histograms. There is small side peak at about vcal=56 with ~100 pixels.
But this should not be a too big problem?

Also the Pulse height map looks good and the reconstructed pulse height at vcal=70
gives vcal=68.1 with rms=4.2, see the attached plot.

So I conclude that this module is fine.

259

Mon May 11 14:40:16 2020

M1582

On Friday I have tested the module M1582 at room temperature in the blue box.
The report in MoreWeb says that this module has problems with trimming 190 pixels in ROC1.

I see not problem in ROC1. The average threshold is 50 with rms=1.37. Only 1 pixel is in the 0 bin.
See the attached 1d and 2d plots.

Also the PH looks good. The vcal 70 PH map is reconstructed at vcal 70.3 with rms of 3.9.
5159 pixels have valid gain calibrations.

I conclude that this module is fine.
Maybe it is again a DTB problem, as reported by Andrey.
D.

287

Tue Jun 9 17:19:00 2020

gel-pack transfers

The following modules, which were already X-ray tested and came back from ETHZ,
have been transferred to gel_packs:
1623
1630
1632
1634
1636
1637
1639
1640

The following "good" modules have been transferred from gel-packs to frames,
they still have to go to ETHZ for X-ray testing:
1620
1622
1624
1626
1627
1628
1629
1631

D.

290

Tue Jun 16 19:23:37 2020

modules from and to ETH, list 3

The 27 modules from list 2 have been transferred back to PSI.

The following 18 modules have been transferred to ETH:

1605
1606
1607
1608
1609
1610
1612
1614
1615

1613
1618
1619
1629
1622
1624
1626
1627
1628

291

Thu Jun 18 20:04:41 2020

Modules to ETH, transport #6

Today Lea has delivered another group of 18 modules to the ETH:
1604
1603
1602
1601
1600
1599
1598
1597
1596

1629
1631
1641
1642
1643
1644
1645
1647
1648

No modules were brought back, so there are now 18+18=36 modules at ETH.

292

Thu Jul 2 13:48:20 2020

Transport #7 to ETH

Lea brought the last batch of 8 modules to ETH:
1542
1593
1665
1672
1673
1674
1675
1676

295

Wed Jul 29 17:19:43 2020

Change configuration for PH qialification

PhQualification

Preparing the new PH optimization I had to make the following modifications:

1) in elCommandante.ini
change ModuleType definition from tbm10d to tbm10d_procv4
in order to use CtrlReg=17 setting

2) in pxar/data/tbm10d_procv4
change in all dacParameter*.dat files Vsh setting from 8 to 10.

I hope these are the right locations.

D.

304

Fri Oct 23 13:33:04 2020

L1 & L2 modules to go to P5

The following modules will be transported to CERN/P5 as spares.

1) L1
1538
1542
1608
1613

also take a D (broken) module 1671 for setup testing.

2) L2
2278
2160
2258
2293
2078
2026
2122
2155
2035
2036
2298
2244

2269 was at the top of Andrey's list but I just discovered that it does not have the cap,
so we leave it at PSI.

D.K.

305

Fri Nov 6 07:28:42 2020

Dinko Ferencek wrote:

pXar parameter files were converted to POS configuration files by executing the following commands on the lab PC at PSI

cd /home/l_tester/L1_SW/MoReWeb/scripts/
python queryModuleLocation.py -o module_locations.txt -f
python prepareDataForPOS.py -i module_locations.txt -p /home/l_tester/L1_DATA/ -m /home/l_tester/L1_DATA/WebOutput/MoReWeb/FinalResults/REV001/R001/ -l /home/l_tester/L1_DATA/POS_files/Folder_links/

cd /home/l_tester/L1_SW/pxar2POS/
for i in `cat /home/l_tester/L1_SW/MoReWeb/scripts/module_locations.txt | awk '{print $1}'`; do ./pxar2POS.py -m $i -T 50 -o /home/l_tester/L1_DATA/POS_files/Configuration_files/ -s /home/l_tester/L1_DATA/POS_files/Folder_links/ -p /home/l_tester/L1_SW/MoReWeb/scripts/module_locations.txt; done

The POS configuration files are located in /home/l_tester/L1_DATA/POS_files/Configuration_files/

Dinko

I have finally looked more closely at the file you have generated. They seem fine exept 2 points:
1) Some TBM settings (e.g. pkam related) differ from P5 values.
This is not a problem since we will have to adjust them anyway.

2) There is one DAC setting missing.
This is DAC number 13, between VcThr and PHOffset.
This is the tricky one because it has a different name in PXAR and P5 setup files.

DAC 13: PXAR-name = "vcolorbias" P5-name="VIbias_bus"

its value is fixed to 120.

Can you please insert it.
D.

Mon Feb 24 13:53:32 2020

Setup changes in week 8

Setup changes in week 8

Silvan considered the single-module gluing approach suboptimal. It was changed:
- we switched sides, the cap gluing is now done on the window-side of the aisle
- there are now two jigs set up for cap gluing

The procedure itself for gluing did not change: Apply the stamp with the glue to module 1, release, put module 1 into its jig, insert module 2 into gluing jig, apply stamp (because of the glue fluidity, the marks from module 1 will have disappeared and there is no need to re-apply any glue to the stamp). The caps are lowered onto the modules only once the glue has been applied to both modules.

Silvan recommended applying the weight of only one copper(?) bar, to be placed centered on the top of the z-stage.

Silvan also recommended against using ethanol. Instead we are supposed to use aceton. As a result, the plastic tweezers should not be used anymore, but rather metallic tweezers.

Finally, Silvan changed the Peltiers, and applied isolation inside Red October. We observed cooling times from 10C to -20C of 6 minutes.

Mon Feb 24 14:01:29 2020

Cap gluing in W8

In week 8 the following modules were glued and tested:

M1581
M1582
M1583
M1584
M1585
M1586
M1587
M1588
M1589
M1591

M1590 was not glued last week because it had one HV bond broken (Silvan fixed this on Monday, 20/02/24).

M1586 initially had problems with the readout after gluing, but this was fixed by porperly closing the MOLEX connector. In the reception test, M1586 was graded 'B'.

All other modules were graded 'A' in the reception test.

Overall the double-gluing setup works very well. The modules above were glued basically in one day.

Tue Feb 25 17:11:18 2020

Modules glued and tested Feb 24/25

Modules glued and tested Feb 24/25

The letters after the module name indicate the reception test grade.

M1590 A (after Silvan fixed broken HV bond)
M1592 A
M1595 B
M1596 A
M1597 A
M1599 C (leakage current: 16uA)
M1600 A

M1598 had a broken clock wire bond, was diagnosed by Wolfram and fixed by Silvan. Will be processed together with the other modules that had problems after taking them out of the storage box (M1593 and M1594), in case they can be fixed.

Wed Feb 26 10:34:55 2020

Fulltest after Peltier replacement

On 2020/02/25, after SIlvan had replaced the Peltiers, I did a fulltest. Here the summary:

Finished all tests. Summary of test durations:
Fulltest@-20 127 min 5 sec
IV_TB0@-20 4 min 23 sec
IV_TB1@-20 5 min 1 sec
IV_TB2@-20 5 min 1 sec
IV_TB3@-20 5 min 1 sec
Cycle 39 min 23 sec
Fulltest@-20 121 min 6 sec
IV_TB0@-20 4 min 23 sec
IV_TB1@-20 5 min 1 sec
IV_TB2@-20 5 min 1 sec
IV_TB3@-20 5 min 1 sec
Fulltest@10 115 min 17 sec
IV_TB0@10 4 min 42 sec
IV_TB1@10 5 min 5 sec
IV_TB2@10 5 min 5 sec
IV_TB3@10 5 min 5 sec
--------------------------------------------------
total 461 min 48 sec

All modules were graded 'B' (I think because of the noise mean being to high in a few chips per module)

Thu Feb 27 09:47:51 2020

Fulltest on 2020/02/26

On Wednesday, 2020/02/26 the following modules went through the full qualification procedure:
M1581
M1582
M1583
M1584

All received grade B (from a cursory glance due to mean noise).

Fri Feb 28 17:33:16 2020

Fulltests on 2020/02/28

M1589 B
M1590 C (gain issues?)
M1591 C (noise issues?)
M1592 B

Mon Mar 2 17:39:36 2020

Fulltests on 2020/03/02

Modules tested:
M1595 C
M1596 B
M1597 C
M1600 C

Andrey and I suspect that these grades are driven by a bad PH optimization.

Wed Mar 11 15:34:57 2020

M1586: issues with MOLEX?

Module M1586 had passed the full qualification on 20/02/27. I had had to re-insert the cable in the Molex connector for it to become programmable.

On 2020/03/09, I tried to re-test M1586, but it was not programmable. Visual inspection revealed nothing to me. I did re-insert the cable once again, but this time this did not help.

Maybe one should try again re-inserting the cable.

Maybe these issues are an indication that the module (MOLEX) is flaky.

147

Fri Mar 27 10:54:04 2020

Issues with pc11366 on March 27

On March 27 I had a lot of troubles getting the full qualification up
and running.

The problems were (1) strange error messages from pxar core, (2)
problems with USB connections to the DTBs, (3) loads of (intermittent)
data transmission errors from the DTBs, and (4) complaints about missing
(system) libs.

In the end I did killall firefox and (out of superstition) pkill compiz.

Then it worked again.

163

Tue Mar 31 09:29:02 2020

exchanged adapter for DTB_WXC03A

I did not manage to get any r/o from the module connected to that adapter, also after exchanging the module.

So I exchanged the module adapter with the one from the blue box, and all issues with r/o were gone (with both modules)

Of course, the apparently flaky adapter from WXC03A seems to be working again/now in the blue box.

Just fyi.

164

Tue Mar 31 10:32:32 2020

M1637

Maybe M1637 has an issue: It got stuck twice at the same place with

[09:53:16.043] <TB0> INFO: PixTestReadback::CalibrateIa()
[09:53:16.043] <TB0> INFO: ----------------------------------------------------------------------
[09:53:17.924] <TB0> ERROR: <datapipe.cc/CheckEventID:L486> Channel 6 Event ID mismatch: local ID (22) != TBM ID (23)
[09:53:17.924] <TB0> ERROR: <hal.cc/daqAllEvents:L1701> Channels report mismatching event numbers: 23 22 22 22 22 22 22 22

After that, the DTB is unresponsive and elCommandante loses it. (The printout above is from the second testrun. I had restarted the complete fullqualification after realizing that DTB0 was 'missing' and checking manually that M1637 was properly connected).

I hope elCommandante manages this gracefully.

167

Tue Mar 31 17:36:36 2020

exchanged adapter for DTB_WXC03A

Urs Langenegger wrote:

Today I have to reconnect a few times modules to the module adapter in the blue box.
It looks like the Molex connector is a bit damaged. One should be very careful while connecting a cable to this Molex connector.

237

Wed Apr 29 08:48:06 2020

M1539

M1539 showed no readout. I tried, all without success,
- reconnecting the cable to the adapter multiple times
- connecting to the adapter in the blue box
- reconnecting the cable to the MOLEX on the module

294

Fri Jul 10 11:24:37 2020

proc600V3 modules

This week I tested 12 modules built with proc600v3. The module numbers are M1722 - M1733.
The results are summarized at the usual place:
http://cms.web.psi.ch/L1Replacement/WebOutput/MoReWeb/Overview/Overview.html

Cheers,
--U.

143

Thu Mar 26 15:37:18 2020

Urs

M1606

attached find the threshold difference distributions for all four trimbits for all ROCs

157

Mon Mar 30 15:40:52 2020

Urs

FT of M1629, M1630, M1631, M1632

M1630 is interesting because (I am using my terminology in the following) for the test at T=+10C ROC1 fails the PH optimization test and by consequence the gain/pedestal test is also failed.

The PH optimization test is failed because the minimum pixel on which the test is based is a 'dead' pixel (according to the PixelAlive test), but unfortunately has hits in the initial PH map. As a result the phscale and phoffset for this ROC are not optimal and this is seen in the gain/pedestal fits.

Please find the plots attached from the T=+10 tests.

Tue Aug 6 15:15:00 2019

Activities 19.6.-4.7.2019.

17.6.-21.6.

Andrei and Matej were at PSI working on the cap gluing setup
Andrei cap glued M2029 (Box9,T110)
Matej cap glued M2331
M2331 was tested before and after cap gluing
M2233 was taken to Zagreb for irradiatio, cable of the mdule is in Box4, T040

1.7.

HDIs 3019 and 3020 were tested and both failed, results in the HDIresults
Performed characterization on one v4 chip, it will be irradiated in Zagreb and restested

2.7.

Silvan bonded the M1520 so Andrei and Matej performed full qualification on it (-20 and +10) and it took 5 hours

3.7.

Glued caps on 3 dummy modules with 3 different glues.
HDI 1317-3-008 with two component epoxy glue
HDI 1317-3-010 with SG-20 silicon glue
HDI 1317-3-004 with WS-200 silicon glue

4.7.

3 dummy modules taken to Zagreb for irradiation by Matej
M1508 was also taken

Tue Aug 6 16:01:17 2019

Wrong dac settings - elComandante

If it seems that elComandante is taking wrong dac settings for tests like Reception test or full qualification, one should remember that it does NOT read values from module specific folders like "M1523", but rather from tbm-specific folders like "tbm10d". The folders from which the dacs are taken are listed in "elComandante.config", the lines which look like "tbm10d:tbm10d".

Tue Aug 6 16:06:45 2019

Vsh and ctrlreg for v4 chips

The recommended default value for Vsh is 8, but the current version of pXar has it as 30. One should remember this when making new configuration folders from mkConfig. The recommended value of ctrlreg is 17.

Wed Aug 7 11:49:07 2019

HDI glue irradiation tests

Six different glues were used to glue the cap on six dummy modules. They were all irradiated to 120 MRad in Zagreb after which they were taken to PSI and tested with the tweezers under the microscope.

Standard glue (Dow-cornig), used in CMS - it became "two-component". Solid part on the capacitors and the liquid part on the cap. Surface tension was actually holding the cap quite strongly. Silvan deemed it the second best glue

Two component epoxy adhesive - the strongest glue out of all tested. The only drawback of it was that during the application of the glue to the capacitors, it left puddles around some of them since it is quite liquid before curing. This might have been prevented if we had a proper glue stamp. The glue stamp used at the time contained too much excess glue, and if another one was printed, with small grooves where the capacitors are, the amount of glue transferred should be lower and no puddles should appear. This glue was graded the best of all the tested glues. EDIT: the glue stamp with smaller grooves was printed and now there are no puddles anymore.

SG-20 (black) - sticks, but not really well. It was also quite soft and malleable. Because it doesn't stick very well, it is not recommended for gluing the cap.

WS-200 - almost identical to the SG-20, just a different color.

Terosan MS939 - doesn't stick, not recommended.

Ergo 6521 - doesn't stick, not recommended.

Took photos of them under the microscope. There are two photos for each glue. First one is while the cap is at rest, while on the second there is an upwards force applied with the tweezers. All of them except the two component epoxy adhesive detached when force was applied. With the epoxy adehsive, the module started lifting, but the glue held.

Wed Aug 7 12:24:14 2019

Activities 31.7.-9.8.2019.

Documentation

31.7.

Matej tested 4 HDIs (8030-8033 and 8010). 8010 had a faulty TBM which was replaced, however, during subsequent tests, other TBM (the one which was working fine!) started sparking during HV test. Burn marks visible under the microscope in the top-right corner of the right TBM, between wire-bond pads. Other HDIs were fine and were glued to 2 v4 modules (1523 and 1529) and 1 v3 module (1526).

1.8.

Prepared the software and hardware for the full-qualification of multiple modules in parallel. Ran reception test on M1520 and M1522.

2.8.

Ran full qualification on M1520 and M1522. It consisted of qualification at -20, IV@-20, 5 cycles between -20 and +10, qualification at -20, IV@-20, qualification at +10 and IV@+10. This set of tests will from now on simply be called "full qualification". For some reason, second qualification at -20 failed. M1522 shows significantly high leakage current compared to previous tests

5.8.

Boxes 2 and 9 have been filled with L1 module trays. Each tray can hold 4 modules and each box contains 13 trays. Box 9 filled with humidity collecting balls. M1520 and M1522 placed in the box 9 which is the box that should be filled first. Reception test was run on M1523 and M1526. There were a lot of issues with M1523 in the address decoding test affecting other tests as well. What was found out is that the third dac, Vsh, was set to the wrong value by default if one was making module configuration folder using mkConfig script. It was 30 and should be 8. This should be propagated in pXar later on.

6.8.

Started running reception tests with 3 modules (1523,1526,1529) in parallel, but with the corrected value of Vsh (8). Several issues were observed.

First, one of the DTBs had a connection timeout which appeared every time we tried the reception test. Changing the DTB didn't work. This was solved by plugging the USB cable on the PC side to a different USB slot
Second, pretest in the reception test did timing test which we deemed unnecessary so it was removed.
Third, results for 1523 and 1529 didn't look correct. It was as if there was a problem with either timing or dac values. Comparing the logfiles of the test and the pxar files, it was found out that the timing settings were correct. The issue was in the dac settings taken by elComandante. What we forgot was that elComandante does NOT take dac settings frmo the module folder, but rather from generic tbm folders like tbm10d. This is set in the elComandante.config. The issue was that tbm10d folder had wrong values for Vsh (30, instead of 8) and ctrlreg (0 instead of 17). After this correction reception tests ran fine.

PixPhOptimization.cc was edited to correct the starting values of the parameters for the PhOptimization test. The old values were sometimes causing the tests to algorithm to fail even though the chip was fine.

Started full qualifications of all 3 modules, but there were some issues with the coldbox. It was noticed when it was not cooling at the start of full characterization. The warning on the coldbox interface read "Geber-stillstand". Danek managed to make it go away by stopping the program, even though no program was being run.

Finally, full qualification started around 15:00, finished at 22:40.

M1523 failed first qualification at -20, but finished other two qualifications. IV curves for that module were taken while M1529 was stuck and connected to HV as well meaning that the readings are actually sum of IVs from these two modules.
M1529 failed first qualification at -20, few seconds apart from M1523 failure. One of them failed during the Gain Pedestal, other one on PhOptimization. It also failed other two qualifications (PhOptimization and GainPedestal)
M1526 had no issues in qualification, however, it was graded C. One chip was problematic in most of the tests, while 4 others had issues with TrimBits. This is a bit suspicious so we'll investigate it further.

7.8.

Investigating problems with qualifications on testboards 0 and 1. We noticed that it happens during PhOptimization or GainPedestal. Also investigating why M1526 failed trimbits test on several ROCs. The cause was identified as a memory leak (several Gbs!) during a PhOptimization scan. Separate log made for this.

8.8.

Started full qualification WITHOUT PhOptimization at 9:15 on modules 1523, 1526, 1529. Finished after 8 hours. 1529 had a reception test error during fulltest@+10, other tests ran fine. Glued protection caps on 4 modules - 1504,1505,1509,1520 using 2 component epoxy adhesive.

9.8.

Running fulltest@-20 and +10 for modules 1504, 1505, 1520. The plan is to irradiate them in Zagreb and evaluate the changes induced by radiation.

Wed Aug 7 18:01:06 2019

PhOptimization problem

August 7 - we found out that PhOptimization algorithm starts leaking memory if it fails to find proper values. If running one setup, the test might go through. However, if multiple modules are tested in parallel and they all start leaking memory at the same time, the system will kill one testboard process (or two if necessary), causing the unlucky testboard(s) to get stuck (powered on, HV on, but no tests running) until they are reset. Current solution to this problem is to simply omit the PhOptimization if the fulltest procedure. We left gainPedestal, since it will use default Ph dac values.

Tue Sep 10 15:18:20 2019

Modules 1504, 1505, 1520 irradiation report

Modules 1504, 1505 and 1520 were taken to Zagreb for irradiation on 13.08.19. The goal was to check the v4 behavior after irradiation. They were irradiated to 1.2 MGy and returned to PSI on 9th of September. Upon testing them at PSI, they all had issues. ROCs on M1504 and M1520 were not programmable at all, changing Vana had no effect. Vana could be set on M1505 while targeting Iana = 28 mA/ROC, however, no timing could be found for it and no working pixel could be found. Andrey and Matej took the modules under the microscope and saw a "greenish" deposits on HDI metal pads of unknown origin. There was also a bit of liquid near the HDI ID on M1520, but not on others. The residue could be shorting some pads causing issues on the module. It is still unclear whether the residue comes from the HDI, cap or is it introduced during irradiation by something in Zagreb.

Thu Sep 12 10:32:27 2019

Activities 10.9.-19.9.2019.

10.9.
Modules 1504,1505,1520 brought to PSI after irradiation to 1.2 MGy in Zagreb. All 3 were not working when put under test with pXar. 1504 and 1520 couldn't set Vana at all. 1505 could set Vana, but timing couldn't be found and pixels weren't responding. Inspection under the microscope showed some "white-greenish" deposits on the metal pads on the HDIs which are most likely shorting the ROC pads causing the module to be non-responsive

11.9.
Removed cap from 1504 to take a better look under the microscope and some photos. The good news is that the glue holds really well. The deposition looks like a crystal growth. It reminded Danek of the humidity issues they had at CERN in 2014. Touching it with a needle showed that it is not completely solid, like "wet-snow", however, on some pads, the deposition was more solid. There was also small transparent liquid deposits observed. Prepared 3 modules for cap gluing training; 1047, 10X1, 10X2. Only 1047 works (partially) and Reception test was performed on it. Two gluing jigs prepared in the assembly lab.

12.9.
Ran reception on 1509, it will go to ETH for irradiation (not really sure if it's ETH). Further tested modules 10X1 and 10X2. X2 should have hubID set to 17,25 after which the ROCs are programmable, however, there is no readout. X1 should have hudID 31,30; but only TBM with hubID 30 works, ROCs are programmable, but no readout as well. Glued protection cap to a module 10X1. Roughly 4 minutes after the start of mixing, the glue starts to become fairly solid which gives us a constraint on the time it takes us to apply it to a module.

13.9.
Glued caps on 1508 and 1047 using the jig further from the entrance door of the assembly lab. 1508 was glued first and had an asymmetrically placed cap (one side of the cap quite close to the wirebonds). The stage was adjusted so the cap glued to 1047 was properly placed. An unused HDI was found 8-011, it was tested and found to be working until the HV test where it started sparking at -1000V, similar to 8-010. There are burn marks visible under the microscope on the same position as the ones on 8-010.

16.9.
Investigated further the sparking issue with the HDIs. Ed Bartz took a look at the photos and identified that the damaged part of the TBM is the protection diode of pin 46.

17.9.
Equipped the coldbox with 4 DTBs which makes it possible to do qualification on 4 modules at the same time. Investigated phase2 ROC-HDI gluing jig.

Wed Sep 18 15:45:45 2019

HDI sparking under HV test

HDI 8010 passed all test except the HV. Under the HV test, some sparks could be heard and even seen by eye on the right TBM (closer to the HV pad!). Testing it again, showed that it was indeed damaged by the spark. Sparking craters could be seen with the microscope between pads 4 and 5 of the TBM.

The same thing happened a month later, on HDI 8012. Again, it passed all the test until HV when sparks appeared and damaged the TBM, and, curiously enough, the damage was on the same position on the TBM as with 8010.

HDI 8011 was put under the HV test as well, however, this one didn't have any wires bonded to the TBMs because gold plating was missing on several of the wire bonds. Sparking could be heard, but we couldn't located where the damage happened. There is one potential candidate, shown on the photo attached to this log entry.

On 18.09. a possible explanation for the sparking was thought of. Wolfram and Matej put one of the "faulty" HDIs under HV test and noticed that sometimes a spark could be seen between the HDI handle and the aluminum base plate! It looks like the HV pin rests near the border of the HV pad and the ending of the HDI. Because of that, there is a discharge from the HV pin to the HDI handle. The discharge connects to the ROC wire bond pads closest to the HV pad providing a way to affect the TBM! This is further strengthened by the fact that the damaged TBM pad is connected to the first four ROCs closest to the HV pad. To test this, we put kapton tape on the HDI handle, near the HV pad and tested the HDI again and there were no sparks.

The HDIs to be used for L1 replacement are slightly longer to prevent HV jumping to the sensor. Incidentally, this might also solve the sparking issue. If the issue is not solved on the longer HDIs, HDI holders will have to be further isolated in the region close to the HV pad.

Thu Sep 19 14:20:07 2019

Activities 10.9.-19.9.2019.

Matej Roguljic wrote:

18.9.
Decided on a final HDI irradiation test. We will have an HDI without components in a bag filled with nitrogen (N2 bag), HDI with components in N2 bag, Dummy module (HDI with cap glued on it) in N2 bag, Dummy module not in an N2 bag and and an HDI with components cleaned by Silvan not in an N2 bag. The idea is to expose them to irradiation in a dry environment to check if there will be weird deposits on the metallic pads of the HDI. With the 2 samples not in an N2 bag we will see if there are some deposits again. To do so, before irradiation we will inspect them under a microscope and take photos and inspect them immediately after irradiation.

Furthermore, a possible explanation for the HDI sparking problem was devised and solutions, if necessary. It is described in more details in a separate log entry.

Thu Sep 26 17:38:07 2019

Activities 25.-26.9.

It was planned to test a batch of HDIs these 2 days but the shipment was late. Dinko and Andrei were shown the HDI testing procedure. Matej further worked on the script used to test the HDIs. Dinko glued caps on 2 dummy modules and on M1522. New HDIs arrived on 26th and it looks like the kapton will have to be put underneath them during the testing to prevent sparking.

Mon Oct 28 17:01:05 2019

Investigating the bug with the Keithley client

We took module 1529 and tried recreating the issue observed in the beginning of October. To do this in a reasonable amount of time, a "shorttest" procedure was defined which consists only of pretest and pixelalive. Three runs were taken

Run number 1: Shorttest@10,IV@10
Run number 2: Shorttest@10, IV@10,Cycle(n=1, between 10 and -10), Shorttest@10, IV@10
Run number 3: Shorttest@10, IV@10, Cycle(n=5, between 10 and -10), Shorttest@10, IV@10

In runs 1 and 2, IV was done from -5 to -155 in steps of 10
In run 3, IV was done from -5 to -405 in steps of 10

No issues were observed during the tests themselves.

Running MoReWeb shows the Temperature, Humidity and Sum of Currents plots while individual tests show only Pixel Alive map. IV plot is missing in the MoReWeb output, however, it is present in the ivCurve.log file. Tried investigating why IV is not shown, but couldn't get to the bottom of it.

Tomorrow we'll use M1529 and M1530 in a FullTest to check if the problem would appear.

Tue Oct 29 16:36:23 2019

Assembly lab cap gluing jigs

There are two jigs for cap gluing in the assembly lab, circled in red and blue on the photo in the attachment. The one closer to the door (red) we used to build glue Phase 2 HDI to a dummy sensor and four Phase2 ROCs. Before changing the head, the numbers of the alignment screws were noted.

The jig closer to the doors (red): Top two pins - 5.75; Left pin - 5.5
The jig further from the doors (blue): Top two pins - 5.71, Left pin - 3.25

Tue Oct 29 16:38:32 2019

Investigating the bug with the Keithley client

Matej Roguljic wrote:

M1530 was put in the coldbox and qualification was started, but the pxar output was full of deserializer errors. The continuous stream of errors made the log file pretty large and slowed down the execution of the qualification so it was aborted. Later it was noticed that there are green depositions on the module like it was on the HDIs irradiated at 60Co facility in Zagreb which is probably why there were so many errors.

M1521 was used instead of M1530 along with M1529. Full qualification was launched around 11:00.

Wed Oct 30 10:38:04 2019

Activities 28.-31.10.2019.

28.10. - Investigated a problem with the FullQualification setup, described more in this note
29.10. - Further investigated the aforementioned problem and worked on preparing the jigs for Phase2 dummy module building.
30.10. - Confirmed that the reported issue is not present at the moment on the qualification system. Glued dummy sensor to four phase2 ROCs. Built a FC7 nanoCrate.

Wed Oct 30 16:54:58 2019

Investigating the bug with the Keithley client

Matej Roguljic wrote:

Running full qualification on 30.10. in the morning on modules 1510, 1529, 1521. Fourth module was not included since there was an issue with the 4th DTB which will be investigated late. Like the previous day, the qualification ended around 17:00 without issues. In conclusion, the issue is not present in the qualification setup at the moment.

Thu Oct 31 10:41:12 2019

Issue with DTB_WWVASW

On 29th of October, an issue with DTB_WWVASW was noticed when trying to run FullQualification on 4 modules at the same time. On a tested, working module, setting Vana works but taking tornado plots (setvthrcomp) doesn't as well as other tests. Curiously enough, after switching the DTB with the one used for HDI testing (DTB_WRN13L) the problem was still there. This lead us to believe that the problem was with cables or ground. All cables and the adapter were tested and shown to be working fine. This is when we took a third DTB, put it in the same plae as previously WWVASW and WRN13L, connected all the cables and then the tests worked.

Conclusion of the tests is that DTB_WWVASW and DTB_WRN13L are suffering from the same failure mode. It was not noticed before on the WRN13L because it does not affect the HDI tests. This leaves us with 3 working DTBs for the FUllQualification tests.

Wed Nov 27 08:39:24 2019

Bump bonding test investigation

The BB test on M1534 shows a lot of dead bunmps on C10 and some on C1 and C0 as well. Putting a source on top of the module shows that the pixels for which the test showed dead bumps are still able to read hits from the source. Therefore, it looks like the bumps are working, but the test is reporting them as defective. It turns out that the bumps used by Helsinki are different than the bumps used at PSI so a different test needs to be used. After trying all 4 BB tets in pXar, it was concluded that BB2 is the appropriate test to use.

Thu Nov 28 15:58:20 2019

Activities 26.-29.11.2019.

26.11.

Prepared the tools for RD53A digital module assembly in the assembly lab
Investigated if the latest pXar commit solved the issue with PhOptimization, now called Ph, and it runs successfully.
Investigated the bump bonding issue in which pXar reports bad bumps, but we suspected that was not the case. And indeed, putting a source on top of the chip reported to have bad bumps and taking data shows that there are no bad bumps as reported by pXar. Our conclusion was that the BB test developed by PSI is not applicable for bumps bonded by Helsinki. BB2 seems to be appropriate.

27.11.

Assembled a Phase 2 digital module using good ROCs - 1A48, 1A47, 1A43, 1A42; being ROC 0,1,2 and 3, respectively.
Used X-ray box to confirm our BB findings yesterday and the results agree that there are no defective bumps (as reported by pXar) on M1534.

28.11.

Took a look at the assembled module and it seems that one corner of the HDI was not properly glued. There are also solder marks on a couple of pads so we decided not to wire-bond it.
Measured the alignment of the two assembled module (one module was assembled during my last visit) using the microscope. The discrepancies in the distance between HDI wire-bond pads and ROC wire-bond pads are around 200 microns. If we take the length of the module to be 4cm, this is less than half a degree of tilt.

Worked on presentation reporting these activities for the tracker week.

Mon Jan 20 18:09:50 2020

Activities 20.-31.01.2020.

20.01.

Expanded the HDI testing setup to be able to mount wire-bonded HDIs to HDI handles. Tested 12 HDIs (v7.2), all of them passed the tests. HDIs were given to Silvan and are located in the storage box in the assembly lab. They will be used for module production.

21.01.

Tested 4 HDIs, version 7.1. All passed the electrical tests.
Tested 16 v7.2 HDIs. All but one passed the electrical tests.
Glued a protection cap on a pre-production module (M1535).

22.01.

Investigated problems with full qualification setup, narrowed it down to communication between PC and Keithley.

23.01.

Tested 23 HDIs. One failed the test, rest of them passed.

24.01.

Tested 15 HDIs. All of them passed the tests.
Investigated problems with threshold/trim tests in full qualification when going to -20. Suspecting wrong DACs used in the test.

27.01.

Continuing to investigate problems with threshold in trimming procedures. The problem seems to be in the DAC 'ctrlreg' which was set to 17. Setting it to 9 removed the problem for M1529.

28.01.

Confirmed that 'ctrlreg' works also at room +20 degrees. Did reception test on modules 1529, 1536, 1537 and 1538. After this, caps were glued to modules 1537 and 1538. Did fulltest@-20, IV@-20, fullltest@+10 and IV@+10 for modules 1534, 1532, 1529 and 1536.

29.01.

Ran reeption on moules 1539 and 1540. Ran full qualification on modules 1536, 1537, 1538, 1540. Module 1539 turned out to have broken wirebonds on one TBM, reported this to Silvan and he re-bonded it. Assembled a second cap-gluing station.

Mon Jan 27 17:50:48 2020

Failing modules at -20

Friday, 24.1., it was noticed that the modules are graded as 'C' in similar fashion. Module is graded A/B at +10 degrees and then graded C at -20. The reason for grade C was that the trimming procedure was not successful on at least one of the ROCs on each module. This problem was further investigated on 27.1. Wolfram suspected that the problem lies in zero suppression mode since the pixels with failed threshold comes in groups of four pixels. The corresponding DAC 'ctrlreg' was set, as recommended, to 17, during previous tests and after changing it to 9, the problems at -20 seem to be gone. This will be further investigated.

Thu Jan 30 13:51:03 2020

Fulltest failure - psiAgente

During the first fulltest@-20 of the full qualification procedure for modules: 1539, 1541, 1542 and 1543; an error message popped up "psi Agente: self.Testboards[0] failed - the following boards are busy: TB1(DTB...:M1541), TB2(DTB...:M1542), TB3(DTB...:M1543). This means that the first fulltest for module 1539 was not executed.

Second fulltest seems to be working fine (so far)

Cause of this problem is not clear.

101

Wed Mar 11 17:12:05 2020

HDI testing procedure change

HDI test

There is an additional test that will be used from 11.03. on all HDIs. It involves measuring the voltage between ground and the HV pin of the needle card with the goal of checking whether proper voltage is delivered to the HDI. The HDI testing script has been updated and it now prompts the user to set the voltage to -800 V, measure the voltage on the pin and write this into the test results. After this, another instruction has been added which tells the user to raise the Z-stage (needle card) before setting voltage to -1100 V. This is done to prevent sparking from the HV pad to the HV pin if the alignment is slightly off.

While measuring the voltage on the HV pin, one should keep in mind the proper settings on the Keithley. When the voltmeter probe is connected to the pin, the current reading on Keithley will go up and if the current readout range is low, it will limit the voltage even before hitting compliance. This was observed during the initial testing of the new procedure. Range should be set to maximum (100 muA) and compliance should be set to 105 muA. This is high enough that it is not reached while measuring voltage at -800 V.

102

Wed Mar 11 17:22:57 2020

HDI test

8 HDI tests on 11.3.

I tested 8 HDIs today: 5029,5030,5031,6034,6036,6035,1031 and 1032
All of them tested fine.

104

Thu Mar 12 12:19:43 2020

HDI test

5 HDI tests on 12.3.

I tested 5 HDIs on 12.3. 1030,1029,1040,1039 and 1038. 1039 failed all the electrical tests. All the others passed the tests. HDI 1038 has one wirebond which is connected to the pad on the HDI and then it extends back up (like this: TBM./\.HDI/). The connection is good, but I just want to check with Silvan later if this is a problem.

105

Thu Mar 12 17:21:59 2020

Fulltests on 2020/03/12

Modules tested:
M1557 C (trim bits at -20)
M1558 C (gain at -20)
M1590 B (still graded C in moreweb because of the previous full qualification, this should be corrected)
M1600 B (still graded C in moreweb because of the previous full qualification, this should be corrected)

107

Sat Mar 14 17:08:14 2020

Fulltests on 2020/03/13

Modules tested:
M1591 C (gain at -20)
M1595 B
M1597 B
M1598 B

108

Mon Mar 16 10:05:23 2020

PhQualification change

Urs made a change in pXar, in the PhOptimization algorithm. One of the changes is in the testParameters.dat where vcalhigh is set to 100 instead of 255. This was implemented on the PC used to run full qualification. A separate procedure for elComandante was created, "PhQualification.ini", which runs pretest, pixelalive, trimming, ph and gainpedestal. This procedure will need to be run on all the modules qualified before this change was made and later merged with previous full qualification results.

109

Mon Mar 16 11:04:41 2020

PhQualification 14.-15.3.

PhQualification

I ran PhQualification over the weekend with changes pulled from git (described here https://elrond.irb.hr/elog/Layer+1+Replacement/108).

14.3. M1554, M1555, M1556, M1557

First run included software changes, but I forgot to change the vcalhigh in testParameters.dat
The summary can be seen in ~/L1_SW/pxar/ana/T-20/VcalHigh255 (change T-20 to T+10 to see results for +10 degrees)

Second run was with vcalhigh 100.
The summary can be seen in ~/L1_SW/pxar/ana/T-20/Vcal100

15.3. M1558, M1559, M1560

I only ran 3 modules because DTB2 (WRE1O5) or its adapter was not working. Summary is in ~/L1_SW/pxar/ana/T-20/Vcal100

The full data from the tests are in ~/L1_DATA/MXXXX_PhQualification_...

110

Mon Mar 16 15:17:18 2020

M1601 and M1602

Andrey and I assembled modules 1601 and 1602 on Friday, 13.3. On Monday, 16.3. I ran reception on them (both graded A) and then glued protection caps on them.

111

Mon Mar 16 15:20:03 2020

PhQualification

PhQualification on 16.3.

PhQualification was run on modules M1561, M1564, M1565, M1566.

243

Thu Apr 30 16:47:00 2020

MoReWeb empty DAC plots

Some of the DAC parameters plots were empty in the total production overview page. All the empty plots had the number "35" in them (e.g. DAC distribution m20_1 vana 35). The problem was tracked down to the trimming configuration. Moreweb was expecting us to trim to Vcal 35, while we decided to trim to Vcal 50. I "grepped" where this was hardcoded and changed 35->50.

The places where I made changes:

Analyse/AbstractClasses/TestResultEnvironment.py
'trimThr':35
Analyse/Configuration/GradingParameters.cfg.default
trimThr = 35
Analyse/OverviewClasses/CMSPixel/ProductionOverview/ProductionOverviewPage/ProductionOverviewPage.py
TrimThresholds = ['', '35']
Analyse/OverviewClasses/CMSPixel/ProductionOverview/ProductionOverviewPage/ProductionOverviewPage.py
self.SubPages.append({"InitialAttributes" : {"Anchor": "DACDSpread35", "Title": "DAC parameter spread per module - 35"}, "Key": "Section","Module": "Section"})

It's interesting to note that someone had already made the change in "Analyse/Configuration/GradingParameters.cfg"

301

Wed Sep 2 16:09:24 2020

Modules for P

M1595 switch with M1558

Module 1595 was foreseen to go on the inner ladder 5, position -2 (negative two). During the pre-installation test, we saw it had a high leakage current, ~8 microAmps at room temperature. Therefore, we decided to place module 1558 in its place instead of it.

Tue Aug 6 14:30:00 2019

Documentation

Jumo Imago 500 manuals

Manuals can be found at https://www.manualslib.com/products/Jumo-Imago-500-8786441.html

Thu Sep 19 00:52:24 2019

Modules 1504, 1505, 1520 irradiation report

Andrey Starodumov wrote:

It was discovered that these modules were stored in Zagreb in the climatic lab where T was about +22C. These modules have been transported from Co60 irradiation facility to the lab on open air with T>30C and RH>70%. Water in the air under the cap condensated on the surface of HDIs and diluted residuals (from soldering, passivation etc), that after remains liquid or crystallised.
The irradiation itself does not course any damage. This is also confirmed by the fact that after two previous irradiations in Jan and Jul 2019 of modules and HDIs, samples remained in a good shape without any residuals on HSI surfaces. These samples have been kept in an office where T and RH were similar to the outside and not in the climatic lab.

We consider the the case is understood and closed.

Just to clarify. The modules were not transported from Co60 the irradiation facility to the lab in open air but inside a closed Petri dish. Otherwise, there would be no risk of water condensation if the air surrounding modules was allowed to quickly mix with the air-conditioned lab air. Here the problem arose from the fact that it was not only the modules that were brought inside the lab but they were brought inside a pocket of the outside air. A closed Petri dish is not airtight but it significantly reduces mixing of the air inside the Petri dish with the surrounding lab air making it slower than the rate at which the Petri dish and the module inside it were cooling down once brought inside the lab. This could have led to water condensation if the pocket of air trapped inside the Petri dish was warm and humid and had a dew point above the lab air temperature. To prevent this from happening, the solution should be relatively simple and it would be to open the Petri dish and uncover modules before bringing them inside the lab. That way the exchange of air will be faster and the risk of condensation will be basically gone because a warm module will be quickly surrounded by the lab air which will not condense on a warmer surface.

However, there is an additional twist in this particular case. On Aug. 22, when the modules were brought in the lab, there was a thunderstorm in Zagreb in the early afternoon (https://www.zagreb.info/aktualno/zagreb-je-zahvatila-oluja-munje-i-jaka-kisa-nad-vecim-dijelom-grada/227156) with temperature around 21.5 C and RH around 75% at the time the modules were transported (around 15:20), and the whole day was relatively fresh and humid. The outside air on that day would definitely not lead to water condensation in the lab. However, before being brought in the lab, the modules were sitting in a room in the building where the Co60 irradiation facility is located so the air inside the Petri dish was likely similar to the air inside that room (the modules were sitting there for a while and there was enough time for the air temperature and humidity to equalize) and there was not much time to mix with the outside when being transported from one building to another. Unfortunately, there are no measurements of the air temperature and humidity in that room. However, it is worth mentioning that the previous day, Aug. 21, was not very hot and humid with midday temperature around 26 C and RH around 50%. It is therefore likely that the air inside that room and consequently inside the Petri dish was not very hot and humid, making the hypothesis of water condensation in the lab, if not improbable, certainly less likely.

Either way, more careful handling of modules will be needed.

Thu Sep 26 15:51:30 2019

pXar code in /home/l_tester/L1_SW/pxar/ on the lab PC was updated yesterday from https://github.com/psi46/pxar/tree/15b956255afb6590931763fd07ed454fb9837fc0 to the latest version https://github.com/psi46/pxar/tree/e17df08c7bbeb8472e8f56ccd2b9d69a113ccdc3 which among other things contains updated DAC settings for ROCs.

All the configs will have to be regenerated before the start of the module qualification.

Thu Sep 26 21:48:56 2019

Cap gluing training

Today I performed my first cap gluing. As an exercise it was first done on two dummy modules and later on a pre-production module M1522. Before the cap gluing the module M1522 was visually inspected and a Reception test was run. Before the cap gluing everything looked fine. After the cap gluing the module M15222 was again visually inspected and it looked like wire bonds in one of the corners might have been slightly bent and some glue got deposited on some of the wire bonds. The Reception test will be repeated tomorrow.

Thu Sep 26 22:09:14 2019

DAC configuration update

In accordance with the agreement made in an email thread initiated by Danek, the following changes to DAC settings for ROCs

vsh: 30 -> 8
vclorbias: 30 -> 120
ctrlreg: 0 -> 9

were propagated into existing configuration files in

/home/l_tester/L1_SW/pxar/data/tbm10c/
/home/l_tester/L1_SW/pxar/data/tbm10d/
/home/l_tester/L1_SW/pxar/data/M1522/

on the lab PC at PSI.

It should be noted that ctrlreg was changed to the recommended value for PROC V3. For PROC V4 ctrlreg needs to be set to 17 so this is important to keep in mind when using configuration files and modules built using different versions of PROC.

Fri Sep 27 22:24:31 2019

Activities on 27. 9. 2019.

Today Silvan wire bonded TBMs to the first batch of 8 new HDIs which were then tested. 5 out of 8 HDIs received a passing grade. Of the 3 failing HDIs, 2 had a spark occurring. The final testing results can be found in https://github.com/mroguljic/HDI_L1/blob/bb78cddaacc7488910aa26bbf609fcff3618c597/HDIresults.csv. Andrey and Danek performed a visual inspection of the 2 HDIs that sparked but could not find clear indications of locations where the sparking occurred. For one of the HDIs (V7.1-3-018) there was a possible indication of sparking on one of the TBM wire bonds but for the other module (V7.1-12-036) there was no such indication.
Updated HDI testing instructions and added a note about the SDA2,3 signal polarities being opposite to those of SDA0,1
Matej's latest updates to the automatization.py script were pushed to GitHub (https://github.com/mroguljic/HDI_L1/commit/82b5cbb9e39e7ce7dc146bfa3bab0de75840d30e)
Delimiters in the CSV file containing HDI testing results were changed from commas to semi-colons (https://github.com/mroguljic/HDI_L1/commit/82b5cbb9e39e7ce7dc146bfa3bab0de75840d30e) to allow for the use of commas in the comment column
Added reminder to the automatization.py script to check signal output on both A1- and A1+ (https://github.com/mroguljic/HDI_L1/commit/79b46eec08ad7d705b70f38191555cdf79c44f5e)
Pushed CSV files containing HDI testing results to GitHub (https://github.com/mroguljic/HDI_L1/commit/bb78cddaacc7488910aa26bbf609fcff3618c597)
Reception tests were run for M1522 after cap gluing (see https://elrond.irb.hr/elog/Layer+1+Replacement/19)

Fri Sep 27 23:22:02 2019

Cap gluing training

Attached are the bump bonding threshold maps before and after cap gluing and the (before - after) difference for M1522.

Wed Oct 2 12:41:16 2019

First production modules assembled

First production modules (M1530, M1531, M1532) were built on Monday, Sep. 30 2019.

There was a problem with wire bonding of ROC5 on M1530. There is a small crater on one of the ROC pads which appears to had been created by the BareModule probe card needle.

After initial tests with pXar of the 3 modules, noticed the following:

M1530: data missing from ROCs 4 and 5 (expected based on the wire bond problem on ROC5) but otherwise looks fine
M1531: could set Vana but setvthrcompcaldel and pixelalive not showing any data
M1532: looks fine.

Caps were glued to M1530 and M1532 (damaged cap was glued to M1530) and Reception tests were run before and after gluing. Attached are the bump bonding threshold maps before and after cap gluing and the (before - after) difference for M1530 and M1532.

Wed Oct 2 12:50:52 2019

Problem with elComandante Keithley client during full qualification

Full qualification was attempted for M1532 on Oct. 1. After the second Fulltest at -20 C finished, the Keitley client crashed with the following error

  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 147, in check_busy
    self.check_busy(data[1:])
  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 147, in check_busy
    self.check_busy(data[1:])
  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 147, in check_busy
    self.check_busy(data[1:])
  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 147, in check_busy
    self.check_busy(data[1:])
  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 147, in check_busy
    self.check_busy(data[1:])
  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 147, in check_busy
    self.check_busy(data[1:])
  File "/home/l_tester/L1_SW/elComandante/keithleyClient/keithleyInterface.py", line 139, in check_busy
    if data[0] == '\x11': # XON
RuntimeError: maximum recursion depth exceeded in cmp

Because of this, the IV measurement never started (the main elComandante process was simply hanging and waiting for the Keithley client to report it's ready) and the main elComandante process had to be interrupted.

Sat Oct 5 22:59:58 2019

Problem with elComandante Keithley client during full qualification

A new attempt to run the full qualification for M1532 was made on Friday, Oct. 4, but the Keithely client crashed with the same error message. This time we managed to see from log files that the crash happened after the first IV measurement at -20 C was complete and Keithley was reset to -150 V. Unfortunately, the log files were now saved for the test on Tuesday so we couldn't confirm that the crash occurred at the same point.

Wed Nov 27 18:25:08 2019

pXar code in /home/l_tester/L1_SW/pxar/ on the lab PC was updated on Monday, Nov. 25 from https://github.com/psi46/pxar/tree/e17df08c7bbeb8472e8f56ccd2b9d69a113ccdc3 to https://github.com/psi46/pxar/tree/9c3b81791738e1b8ec7dd9f0d1b68f8800f8416c which pulled in the latest updates to the pulse height optimization test. Today a few remaining updates were pulled in by going to the current HEAD of the master branch https://github.com/psi46/pxar/tree/5d358c5ebbf095a7d118cdde9e1e509c41ccc615.

The sequence of commands to update the code was the following:

cd /home/l_tester/L1_SW/pxar/
git pull origin master
cd build
cmake ..
make -j6 install

Wed Nov 27 21:50:02 2019

Reorganized pXar configuration files for pixel modules

Due to different DAC settings needed for PROC V3 and V4, a single folder containing module configuration files cannot cover both ROC types. To address this problem, the existing folder 'tbm10d' in /home/l_tester/L1_SW/pxar/data/ containing configuration for PROC V4 was renamed to 'tbm10d_procv4' and a new folder for PROC V3, 'tbm10d_procv3', was created. For backward compatibility, a symbolic link 'tbm10d' was created that points to 'tbm10d_procv4'.

The difference in the configurations is in the CtrlReg DAC value which is 9 for V3 and 17 for V4.

The following lines were also added in the [defaultParameters] section in /home/l_tester/L1_SW/elComandante/config/elComandante.conf

tbm10d_procv3: tbm10d_procv3
tbm10d_procv4: tbm10d_procv4

Thu Nov 28 18:23:31 2019

Reorganized pXar configuration files for pixel modules

Dinko Ferencek wrote:

Since the new PH optimization code was added under the already existing PH test, the old configuration files would have to be updated to have the correct parameter values set for the expanded PH test. For this purpose, the old configuration folders were renamed

tbm10d_procv3 --> tbm10d_procv3_old
tbm10d_procv4 --> tbm10d_procv4_old

and new configuration files were re-generated from scratch

./mkConfig -d ../data/tbm10d_procv3 -t TBM10D -r proc600v3 -m
./mkConfig -d ../data/tbm10d_procv4 -t TBM10D -r proc600v4 -m

In addition, in both sets of configurations files, the configuration for the BB2 tab in the pXar GUI was moved from moreTestParameters.dat to testParameters.dat to have the BB2 tab available by default when starting pXar using these configuration files.

Thu Nov 28 18:34:00 2019

Dinko Ferencek wrote:

cd /home/l_tester/L1_SW/pxar/
git pull origin master
cd build
cmake ..
make -j6 install

pXar code updated to the current HEAD of the master branch https://github.com/psi46/pxar/tree/9eb0f3844e9c7f98d7701629c5af339632c5d84a to pick up the latest update that allows running of the fullTest() method of each of the tests from the command line and consequently also from elComandante.

Thu Nov 28 18:58:30 2019

Updated Fulltest configuration

The Fulltest definition used on the lab PC and stored in /home/l_tester/L1_SW/elComandante/config/tests/Fulltest had the following content

pretest
FullTest
bb2
exit

where FullTest is defined in https://github.com/psi46/pxar/blob/9eb0f3844e9c7f98d7701629c5af339632c5d84a/tests/PixTestFullTest.cc#L82-L121. For added flexibility, we would like to have individual tests specified in the definition file. However, by default this implies calling the doTest() method for each of the tests while FullTest actually calls the fullTest() method. For Scurves and GainPedestal the two methods are distinct. After the latest updates from Urs to the pXar code, we are now able to change the definition to

pretest
readback
alive
bb
bb2
scurves:fulltest
trim
ph
gainpedestal:fulltest
exit

Note that readback was placed first because in the past it was observed that pXar had a tendency to crash when this test was run last.

Fri Nov 29 17:06:26 2019

Dropbox folder on CERNBox set up

A dropbox folder for elComandante tar files was set up in my CERNBox and synchronized with the following local folder on the lab PC

/media/disk1/DATA/L1_DATA_Backup/CERNBox_Dropbox/

Note that the following link in the home folder

/home/l_tester/DATA

points to

/media/disk1/DATA/

so an alternative path is

/home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/

The CERNBox client was installed on the lab PC following instructions for Ubuntu at https://cernbox.cern.ch/cernbox/doc/linux.html

Mon Jan 20 13:47:33 2020

Dinko Ferencek wrote:

cd /home/l_tester/L1_SW/pxar/
git pull origin master
cd build
cmake ..
make -j6 install

pXar code updated to the current HEAD of the master branch https://github.com/psi46/pxar/tree/f6e42c17c0bb3a44bdb3fa13d8f8afb6cae62a81 to pick up the latest PH optimization updates.

Mon Jan 20 21:33:20 2020

Latest tests of PH optimization

We did some tests by running the latest version of pXar on modules M1521, M1529, M1534 and M1536, specifically trimming and PH optimization. Of the 4 modules tested, the PH optimization converged only for M1521. We plan to further investigate with Urs' help possible reasons for failed PH optimization.

We also noticed a change in the threshold distribution plot of the latest trim bit test compared to the previous version of the test (see attached plots).

Tue Jan 21 12:19:08 2020

Latest tests of PH optimization

Dinko Ferencek wrote:

The problem was traced down to a wrong value of the vcallow parameter in the PH optimization configuration. The value was 10 instead of 50 expected by the test. It turned out that the optimization converged for M1521 because this is a PROC600V3 module for which the configuration files were regenerated for another reason and vcallow was consequently set to the right value of 50. The other 3 modules are PROC600V4 modules and for them old configuration files with a wrong vcallow value were used.

Wed Jan 22 11:39:48 2020

BB2 configuration

When attempting to run the FullQualification this morning, pXar crashed while running the BB2 test. After some investigation, we realized the source of the problem was the BB2 configuration missing from testParameters.dat. The configuration is available in moreTestParameters.dat but by default the mkConfig script does not put it in testParameters.dat. Since we run BB2 in the FullQualification, the configuration needs to be copied by hand every time the configuration files are regenerated.

Wed Jan 22 11:50:20 2020

Module configuration files for interactive tests

There are two module configuration folders set up for elComandante

/home/l_tester/L1_SW/pxar/data/tbm10d_procv3/
/home/l_tester/L1_SW/pxar/data/tbm10d_procv4/

These folder can also be used for interactive tests with pXar. However, in that case these folders get filled with pxar.log and pxar.root files. For some reason, elComandante when copying the module configuration files also copies all pxar.log files. To prevent unnecessary duplication of junk files, two new module configuration folders were set up for interactive tests with pXar

/home/l_tester/L1_SW/pxar/data/tbm10d_procv3_pxar/
/home/l_tester/L1_SW/pxar/data/tbm10d_procv4_pxar/

The configurations are identical to those used by elComandante. However, one needs to remember to keep the tests folders in sync with the elComandante folders once they get updated. This can be done as follows

rsync -avPSh /home/l_tester/L1_SW/pxar/data/tbm10d_procv3/ /home/l_tester/L1_SW/pxar/data/tbm10d_procv3_pxar/
rsync -avPSh /home/l_tester/L1_SW/pxar/data/tbm10d_procv4/ /home/l_tester/L1_SW/pxar/data/tbm10d_procv4_pxar/

To check if there are any extraneous pXar files present in the configuration folders, run

find /home/l_tester/L1_SW/pxar/data/tbm10d_procv?/ -type f $ -name 'pxar*.log' -o -name 'pxar*.root' $

If any, they can be deleted by running

find /home/l_tester/L1_SW/pxar/data/tbm10d_procv?/ -type f -name $ -name 'pxar*.log' -o -name 'pxar*.root' $ -delete

Thu Jan 23 14:54:45 2020

Keithley exchanged

Yesterday during an attempt to run the FullQualification, elComandante would lose control over Keithley after performing the IV curve measurements and would go in the subsequent Fulltest with HV off. This happened both between the two sets of tests at -20 C and between the tests at -20 C and +10 C. Today we exchanged the old Keithley with a new one and we observe that the communication with this new Keithley is much smoother without any error codes and warning sounds produced by Keithley.

In order to properly communicate with the new Keithley, it was necessary to set its communication mode to RS-232 and BAUD rate to 57600 (https://youtu.be/-5RmguqC7xA). Everything else was left unchanged:

BITS = 8
PARITY = NONE
TERMINATOR = <CR+LF>
FLOW-CRTL = XON-XOFF

Thu Jan 23 15:15:45 2020

Trimming Vcal value changed from 35 to 40

Trimming Vcal value changed from 35 to 40 in the testParameters.dat files stored in

/home/l_tester/L1_SW/pxar/data/tbm10d_procv3/
/home/l_tester/L1_SW/pxar/data/tbm10d_procv3_test/
/home/l_tester/L1_SW/pxar/data/tbm10d_procv4/
/home/l_tester/L1_SW/pxar/data/tbm10d_procv4_test/

This is the value we will use for the FullQualification.

Fri Jan 24 18:12:00 2020

MoReWeb code updates

MoReWeb code updated to include the ROC wafer info in the XrayCalibration and XRayHRQualification pages:
https://gitlab.cern.ch/CMS-IRB/MoReWeb/commit/4de8ea39050600367ae0b8e959fcc00f29be45d8

In addition, two BB2 plots with wrong axis labels were fixed:
https://gitlab.cern.ch/CMS-IRB/MoReWeb/commit/3cfa18fc3aab1a29859ba9f0f81aa0ed4c59d5c8
https://gitlab.cern.ch/CMS-IRB/MoReWeb/commit/bb6627e6ab3f4d08eb54970a7e3d3b751983e15a

Fri Jan 24 18:16:09 2020

Protective cap gluing

Today I additionally practiced protective cap gluing by adding a protective cap to module M1536.

Fri Jan 24 18:22:00 2020

L1 replacement web page updated

This week several new links were added to the main L1 replacement web page located at the following URL
http://cms.web.psi.ch/L1Replacement/

The links point to other pages or information related to the replacement module qualification.

Tue Feb 4 19:35:38 2020

7 modules prepared for transfer to ETH: 1536, 1537, 1538, 1540, 1541, 1542, 1543

The following modules have been prepared today for transport to ETH for x-ray qualification:
1536, 1537, 1538, 1540, 1541, 1542, 1543

The FullQualification tar files for these modules have been copied to the common CERNBox folder shared with ETH.

Tue Feb 4 19:51:10 2020

Cap gluing

Today protective caps have been glued to modules 1545, 1547, 1548, 1549, and 1550.

Tue Feb 4 19:52:22 2020

Blue coldbox setup for reception tests commissioned

Today I continued where Matej left off with commissioning the blue coldbox setup for reception tests. All the necessary software (pXar and elComandante) Matej already installed (just had to add the BB2 section in testParameters.dat). What was a problem in communication with Keithley (Model 2400) which was fixed after changing the BAUD rate in elComandante/keithleyClient/keithleyInterface.py to 19200 (value set in Keithley's communications settings) and changing FLOW-CRTL in Keithley's communication settings to XON-XOFF.

The reception test was successfully run for modules 1545, 1547, 1548, 1549, and 1550 after cap gluing.

Wed Feb 5 16:07:03 2020

Module assembly spreadsheet migrated to Google Sheets

Excel file stored on CERNBox containing the module assembly spreadsheet has been replaced with a Google Sheet https://docs.google.com/spreadsheets/d/12m2ESCLPH5AyWuuYV4KV1c5O_NEy7uP5PQKzVsdr3wQ/edit?usp=sharing. This change should make common editing of the spreadsheet easier.

The new spreadsheet has also been linked from the main L1 replacement web page http://cms.web.psi.ch/L1Replacement/

Wed Feb 5 16:18:30 2020

Module location tracking added to the assembly spreadsheet

To keep track of the module location, two columns were added to the module assembly spreadsheet https://docs.google.com/spreadsheets/d/12m2ESCLPH5AyWuuYV4KV1c5O_NEy7uP5PQKzVsdr3wQ/edit?usp=sharing that contain the dates when a given module is shipped to ETHZ and when it is returned to PSI.

Thu Feb 6 00:45:25 2020

Lost at least one Peltier in the coldbox

During an overnight (Feb. 4-5) FullQualification run with modules 1545, 1547, 1548, and 1549 the coldbox lost the ability to maintain the temperature at -20 C. As can be seen from the attached temperature log, the problems already started during the temperature cycles where the last 2 cycles were noticeably longer than the first 3 and finally during the second Fulltests at -20 C the temperature started rising and stabilized around -10 C. Based on these observations it looked like at least one Peltier stopped working.

Silvan took apart the coldbox and will install new Peltiers. Based on measurements Andrey did with the old Peltiers it indeed looks like one Peltier stopped working but the remaining 3 also appear to be at different stages of degradation.

Thu Feb 6 17:09:50 2020

Gluing stamp improvements

Silvan has made additional grooves on the gluing stamp which now sits stably on top of the module without any rocking motion and also allows better glue distribution to all the HDI components that are supposed to receive the glue.

Thu Feb 6 17:19:12 2020

RT for 6 modules: 1551, 1552, 1553, 1554, 1555, 1556

Today reception test was run for 6 modules and looks OK for all module. The modules were graded as follows:

1551: A
1552: A
1553: B (Electrical grade B)
1554: A
1555: B (IV grade B)
1556: B (IV grade B)

Protective caps were glued to these modules.

Thu Feb 6 17:28:23 2020

New Peltiers installed

Today new Peltiers were installed and tested. Unfortunately, we had trouble reaching -20 C with this new set of Peltiers (they are of different type than the ones that were taken out). After better insulation ofthe sample compartment and lowering the chiller temperature from 11 to 6 C, the coldbox was able to reach close to -19.8 C after about half an hour

After lowering the chiller temperature to 5 C, the coldbox was able to reach -20 C

and hold it for about 1 hour when the test was stopped

The plan is to order a new set of Peltiers that are better suited for our use case and in the meantime we will use the newly installed ones with the chiller temperature set lower.

IMPORTANT: Pipes inside the coldbox were left uninsulated. For now this is OK because of the low relative humidity in winter months but could lead to condensation in warmer part of the year when the relative humidity increases.

Fri Feb 7 19:40:27 2020

RT for 6 modules: 1557, 1558, 1559, 1560, 1561, 1562

Reception test is done and all 6 modules were graded A.

Protective caps were glued to 4 modules: 1557, 1558, 1559, 1560

Fri Feb 7 20:35:44 2020

FT for 4 modules: 1545, 1547, 1548, 1549

1545: C/C/C (-20/-20/+10). Problematic ROC 14: mean noise >320e, many dead trimbits, wide trimmed Vcal threshold distribution. IV: 0.20 uA at +10 C, slope 1.09
1547: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.41 uA at +10 C, slope 1.08
1548: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.13 uA at +10 C, slope 1.10
1549: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.33 uA at +10 C, slope 1.12

Chiller operational procedures:
Chiller temperature was initially set to 4 C and the coldbox was able to reach -19.7 C which is within the require 0.5 C margin and the Fulltest could start. About 1 hours into the Fulltest, it was observed that the coldbox temperature rose to 19.0 C. At that point the chiller temperature was lowered to 3.5 C and the coldbox was able to lower and keep the temperature in the -19.2 to -19.3 C range. Once the test at -20 C were done, for the Fulltest and IV at +10 C the chiller temperature was initially raised to +11 C and after about one hour (midway through the Fulltest), it was lowered to +10 C because at one point the coolant temperature rose to +13 C. The temperature history plot is attached.

Sat Feb 8 20:48:11 2020

FT for 4 modules: 1550, 1551, 1552, 1553

1550: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.45 uA at +10 C, slope 1.09
1551: C/C/C (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits in ROC 13, but trimmed Vcal threshold distribution and distribution of trim bits look OK. IV: 0.33 uA at +10 C, slope 1.09
==> Manually regraded to B
1552: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.23 uA at +10 C, slope 1.10
1553: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs, double-column readout problem in ROC 11 (see attached plot). IV: 0.41 uA at +10 C, slope 1.70

Chiller operational procedures:
Chiller temperature set and kept at 3.5 C throughout the entire test. This was done after observing that during tests at +10 C and with the chiller left at +3.5 C, the Peltiers were mostly inactive and there was no excessive heating occurring. This seems to suggest that the coolant temperature at +3.5 C is a good balance against the ambient temperature and heat produced by working modules to keep the temperature at +10 C. The temperature history plot is attached.

Sun Feb 9 10:52:17 2020

FT for 4 modules: 1539, 1554, 1555, 1556

1539: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.17 uA at +10 C, slope 1.08
1554: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.44 uA at +10 C, slope 1.80
1555: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.99 uA at +10 C, slope 1.30
1556: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.87 uA at +10 C, slope 1.67

Sun Feb 9 12:05:47 2020

Cap gluing

Today protective caps have been glued to modules 1561 and 1562.

Sun Feb 9 18:36:13 2020

Manual grading

The procedure for manual grading is described in https://github.com/psi46/MoReWeb/pull/120.

In short, inside the test folder it is necessary to place a text file called grade.txt that contains just one number representing the manual grade:

1=A
2=B
3=C

Addendum:
It appears that manual grading does not work properly if one reruns MoReWeb with grade.txt added. It is necessary to first delete any entries related to the FullQualification using the following command

python Controller -d

after which you need to specify for which module you want to delete entries, e.g. M1537. Once done, you need to run MoReWeb for this module, e.g.

python Controller -m M1537

Sun Feb 9 19:34:55 2020

FT for 4 modules: 1557, 1558, 1559, 1560

1557: C/C/C (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits, failed trimming and mean noise >300e in ROC 4. IV: 0.20 uA at +10 C, slope 1.08
1558: C/C/C (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits in ROCs 2 and 12, mean noise just above 300e for ROC 2, possibly C* module. IV: 0.27 uA at +10 C, slope 1.07
1559: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.23 uA at +10 C, slope 1.09
1560: B/B/B (-20/-20/+10). Electrical grade is B due to mean noise >200e for some ROCs. IV: 0.24 uA at +10 C, slope 1.09

Tue Feb 11 10:25:32 2020

RT for 6 modules: 1563, 1564, 1565, 1566, 1567, 1568

Feb. 10

1563: Grade C, ROCs 8 and 10 not programmable, no obvious problems with wire bonds
1564: Grade B, I > 2 uA (2.17 uA)

Feb. 11

1565: Grade A
1566: Grade A
1567: Grade C, no data from ROCs 8-11, looks like a problem with one TBM1 core
1568: Grade A

Tue Feb 11 17:56:06 2020

Fulltest failure - psiAgente

During today's full qualification the 2nd fulltest@-20 for TB1 (M1562) failed

psiAgente:  self.Testboards[1] failed- the following boards are busy: TB0(DTB_WRBSJ9:M1561), TB2(DTB_WRE1O5:M1564)

This looks similar to the report made on Jan. 30.

Looking at the pXar terminal window I managed to catch the following output

Thread 1 (Thread 0x7fe20f554b00 (LWP 7987)):
#0  0x00007fe20c4e90cb in __GI___waitpid (pid=17338, stat_loc=stat_loc
entry=0x7ffe23b71480, options=options
entry=0) at ../sysdeps/unix/sysv/linux/waitpid.c:29
#1  0x00007fe20c461fbb in do_system (line=<optimized out>) at ../sysdeps/posix/system.c:148
#2  0x00007fe20e998172 in TUnixSystem::Exec (shellcmd=<optimized out>, this=0x1fa2570) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:2119
#3  TUnixSystem::StackTrace (this=0x1fa2570) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:2413
#4  0x00007fe20e99aa43 in TUnixSystem::DispatchSignals (this=0x1fa2570, sig=kSigSegmentationViolation) at /home/l_tester/root/core/unix/src/TUnixSystem.cxx:3644
#5  <signal handler called>
#6  0x00007fe20d5d6794 in PixTestTrim::trimTest (this=this
entry=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:359
#7  0x00007fe20d5da500 in PixTestTrim::doTest (this=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:132
#8  0x000000000040a3d0 in main (argc=<optimized out>, argv=0x7ffe23b7b828) at /home/l_tester/L1_SW/pxar/main/pXar.cc:376
===========================================================


The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum http://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at http://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#6  0x00007fe20d5d6794 in PixTestTrim::trimTest (this=this
entry=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:359
#7  0x00007fe20d5da500 in PixTestTrim::doTest (this=0xcfe82f0) at /home/l_tester/L1_SW/pxar/tests/PixTestTrim.cc:132
#8  0x000000000040a3d0 in main (argc=<optimized out>, argv=0x7ffe23b7b828) at /home/l_tester/L1_SW/pxar/main/pXar.cc:376
===========================================================

However, looking at the relevant section of the code in tests/PixTestTrim.cc

356  for (unsigned int iroc = 0; iroc < rocIds.size(); ++iroc) {
357    for (int ix = 0; ix < 52; ++ix) {
358      for (int iy = 0; iy < 80; ++iy) {
359        if (thr2[iroc]->GetBinContent(1+ix,1+iy) > initialCorrectionThresholdMax - 2) {
360          thr2[iroc]->SetBinContent(1+ix,1+iy,0);
361        }
362      }
363    }
364  }

there is nothing special there, just getting the histogram bin content. Could this be some random memory corruption issue?

Wed Feb 12 10:43:04 2020

9 modules sent to ETH: 1539, 1545, 1547, 1548, 1549, 1550, 1551, 1552, 1553

Yesterday the following modules were sent to ETH:
1539, 1545, 1547, 1548, 1549, 1550, 1551, 1552, 1553

The FullQualification tar files for these modules have been copied to the common CERNBox folder shared with ETH and the transfer dates have been added to the assembly spreadsheet.

Wed Feb 12 11:15:04 2020

Problem with dead trimbits understood

It was noticed that some modules are graded C because of typically just one ROC having a large number of dead trimbits. One example is ROC 10 in M1537

At the same time, for this same ROC the Vcal Threshold Trimmed distribution looks fine

as well as the distribution of trim bits

It turns out that the trim bit test is failing in this and other similar cases because of the tornado plot that is shifted up more than is typical

In the trim bit test code, the Vthrcomp was raised from 35 to 50 but for cases like this one, this is still not high enough. We therefore further increased the value of Vthrcomp to 70.

Grading procedure:
We will manually regrade to B all those modules graded as C due to dead trimbits provided the Vcal Threshold Trimmed distribution looks fine, the number of trimming problems is below the threshold for grade C (167), the distribution of trim bits looks reasonable, and they are not graded C for any other reason.

Side remark:
It should be noted that the Trim Bit Test distribution and the way the number of dead trimbits is counted will not always catch cases when the trim bit test algorithm failed. For instance, in the MoReWeb output for ROC 10 in M1537 one does not immediately see that there are many underflow entries

which arise from the fact that the untrimmed threshold used in the test is too might (Vthrcomp value too low) and Vcal that passes the threshold is not found and left at the default value of 0

This problem of not spotting the algorithmic failure is particularly severe in the case of ROC 2 of the same module M1537 where it goes completely unnoticed in the summary table

it is very hard to spot from the plot (because there is no statistics box showing the underflow)

but the problem is there for a large number of pixels

Even from the tornado plot one would not expect problems

but this tornado is for one particular pixel (column 12, row 22) and there is no guarantee that for other pixels the trim bit test won't fail.

Wed Feb 12 15:40:13 2020

RT for 3 modules: 1569, 1570, 1571; 1572 bad

1569: Grade A
1570: Grade A
1571: Grade B, I > 2 uA (3.09 uA)

1572 from the same batch of 4 assembled modules was not programmable and was not run through the reception test.

Thu Feb 13 21:44:28 2020

RT for 6 modules: 1573, 1574, 1575, 1576, 1577, 1578

1573: Grade A
1574: Grade B, I(150)/I(100) > 2 (4.63)
1575: Grade C, problem with one TBM core
1576: Grade A
1577: Grade A
1578: Grade A

Fri Feb 14 09:48:27 2020

FT for 12 modules: 1561-1576 (1563, 1567, 1572, 1575 excluded)

Feb. 11

1561: C/B/B (-20/-20/+10). Mean noise >200e for some ROCs, dead trimbits in ROC 6, but trimmed Vcal threshold distribution and distribution of trim bits look OK. IV: 0.26 uA at +10 C, slope 1.07
==> Manually regraded to B
1562: TEST INCOMPLETE: pXar crashed during 2nd Fulltest@-20 (m20_2). For more details, see here.
1564: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.78 uA at +10 C, slope 2.25

Feb. 12

1562: C/C/C (-20/-20/+10). TEST INCOMPLETE: SCurves test failed for all ROCs in all 3 Fulltests with the following error message

ERROR: <PixTestScurves.cc/scurves:L277> no scurve result histograms received?!

Trimming does not work in ROC 2.
1565: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.39 uA at +10 C, slope 1.07
1566: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.37 uA at +10 C, slope 1.07
1568: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.13 uA at +10 C, slope 1.15
1569: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.16 uA at +10 C, slope 1.09
1570: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.18 uA at +10 C, slope 1.08
1571: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.43 uA at +10 C, slope 1.09

Feb. 13

1573: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.25 uA at +10 C, slope 1.09
1574: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs, problems with bump bonding and trimming in ROCs 1 and 2. IV: 0.46 uA at +10 C, slope 2.14
1576: B/B/B (-20/-20/+10). Electrical grade B due to mean noise >200e for some ROCs. IV: 0.23 uA at +10 C, slope 1.09

Fri Feb 14 14:05:16 2020

Production yield so far

Of 41 modules produced and tested so far (1536-1576), 6 modules were found to be bad before or during the reception test, 5 were graded C after the full qualification (one of which is possibly C*) and the remaining 30 modules were graded B (of which 4 were manually regraded from C to B).

The overall yield for good modules (A+B/Total) produced so far is (30/41)=73%

Fri Feb 14 14:40:59 2020

RT for 2 modules: 1578, 1580

1579: Grade A
1580: Grade A

244

Thu Apr 30 17:24:57 2020

MoReWeb empty DAC plots

Matej Roguljic wrote:

Some of the DAC parameters plots were empty in the production overview page. All the empty plots had the number "35" in them (e.g. DAC distribution m20_1 vana 35). The problem was tracked down to the trimming configuration. Moreweb was expecting us to trim to Vcal 35, while we decided to trim to Vcal 50. I "grepped" where this was hardcoded and changed 35->50.

The places where I made changes:

Analyse/AbstractClasses/TestResultEnvironment.py
'trimThr':35
Analyse/Configuration/GradingParameters.cfg.default
trimThr = 35
Analyse/OverviewClasses/CMSPixel/ProductionOverview/ProductionOverviewPage/ProductionOverviewPage.py
TrimThresholds = ['', '35']
Analyse/OverviewClasses/CMSPixel/ProductionOverview/ProductionOverviewPage/ProductionOverviewPage.py
self.SubPages.append({"InitialAttributes" : {"Anchor": "DACDSpread35", "Title": "DAC parameter spread per module - 35"}, "Key": "Section","Module": "Section"})

It's interesting to note that someone had already made the change in "Analyse/Configuration/GradingParameters.cfg"

As far as I can remember, the changes in Analyse/AbstractClasses/TestResultEnvironment.py, Analyse/Configuration/GradingParameters.cfg.default and Analyse/Configuration/GradingParameters.cfg were there from before, probably made by Andrey. It is possible that you looked at the files when I was preparing logically separate commits affecting the same files which required temporarily undoing and later reapplying some of the changes to be able to separate the commits. The commits are now on GitLab https://gitlab.cern.ch/CMS-IRB/MoReWeb/-/commits/L1replacement, specifically:

435ffb98: grading parameters related to the trimming threshold updated from 35 to 50 VCal units
1987ff18: updates in the production overview page related to a change in the trimming threshold

253

Thu May 7 00:10:15 2020

MoReWeb empty DAC plots

Andrey Starodumov wrote:

Matej Roguljic wrote:

Analyse/AbstractClasses/TestResultEnvironment.py
'trimThr':35
Analyse/Configuration/GradingParameters.cfg.default
trimThr = 35
Analyse/OverviewClasses/CMSPixel/ProductionOverview/ProductionOverviewPage/ProductionOverviewPage.py
TrimThresholds = ['', '35']
Analyse/OverviewClasses/CMSPixel/ProductionOverview/ProductionOverviewPage/ProductionOverviewPage.py
self.SubPages.append({"InitialAttributes" : {"Anchor": "DACDSpread35", "Title": "DAC parameter spread per module - 35"}, "Key": "Section","Module": "Section"})

It's interesting to note that someone had already made the change in "Analyse/Configuration/GradingParameters.cfg"

I have changed
1)StandardVcal2ElectronConversionFactorfrom 50 to 44 for VCal calibration of PROC600V4 is 44el/VCal
2)TrimBitDifference from 2 to -2 for not to take into account failed trim bit test that is an artifact from trimbit test SW.

1) is committed in 74b1038e.
2) was made on Mar. 24 (for more details, see this elog) and is currently left in Analyse/Configuration/GradingParameters.cfg and might be committed in the future depending on what is decided about the usage of the Trim Bit Test in module grading

$ diff Analyse/Configuration/GradingParameters.cfg.default Analyse/Configuration/GradingParameters.cfg
45c45
< TrimBitDifference = 2.
---
> TrimBitDifference = -2.

There were a few other code updates related to a change of the warm test temperature from 17 to 10 C. Those were committed in 3a98fef8.

254

Thu May 7 00:27:41 2020

Comment about TrimBitDifference and its impact on the Trim Bit Test

To expand on the following elog, on Mar. 24 Andrey changed the TrimBitDifference parameter in Analyse/Configuration/GradingParameters.cfg from 2 to -2

$ diff Analyse/Configuration/GradingParameters.cfg.default Analyse/Configuration/GradingParameters.cfg
45c45
< TrimBitDifference = 2.
---
> TrimBitDifference = -2.

From the way this parameter is used here, one can see from this line that setting the TrimBitDifference to any negative value effectively turns off the test.

More details about problems with the Trim Bit Test can be found in this elog.

255

Thu May 7 00:56:50 2020

MoReWeb updates related to the BB2 test

Andrey noticed that results of the BB2 test (here example for ROC 12 in M1675)

were not properly propagated to the ROC Summary

This was fixed in d9a1258a. However, looking at the summary for ROC 5 in the same module after the fix

it became apparent that dead pixels were double-counted under the dead bumps despite the fact they were supposed to be subtracted here. From the following debugging printout

Chip 5 Pixel Defects Grade A
        total:    5
        dead:     2
        inef:     0
        mask:     0
        addr:     0
        bump:     2
        trim:     1
        tbit:     0
        nois:     0
        gain:     0
        par1:     0
        total: set([(5, 4, 69), (5, 3, 68), (5, 37, 30), (5, 38, 31), (5, 4, 6)])
        dead:  set([(5, 37, 30), (5, 3, 68)])
        inef:  set([])
        mask:  set([])
        addr:  set([])
        bump:  set([(5, 4, 69), (5, 38, 31)])
        trim:  set([(5, 4, 6)])
        tbit:  set([])
        nois:  set([])
        gain:  set([])
        par1:  set([])

it became apparent that the column and row addresses for pixels with bump defects were shifted by one. This was fixed in 415eae00

However, there was still a problem with the pixel defects info in the production overview page which was still using using the BB test results

After switching to the BB2 tests results in ac9e8844, the pixel defects info looked better

but it was still not in a complete sync with the info presented in the FullQualification Summary 1

This is due to double-counting of dead pixels which still needs to be fixed for the Production Overview.

256

Thu May 7 01:51:03 2020

Strange bug/feature affecting Pixel Defects info in the Production Overview page

It was observed that sometimes the Pixel Defects info in the Production Overview page is missing

It turned out this was happening for those modules for which the MoReWeb analysis was run more than once. The solution is to remove all info from the database for the affected modules

python Controller.py -d

type in, e.g. M1668, and when prompted, type in 'all' and press ENTER and confirm you want to delete all entries. After that, run

python Controller.py -m M1668

followed by

python Controller.py -p

The missing info should now be visible.

260

Mon May 11 21:32:20 2020

Fixed double-counting of pixel defects in the production overview page

As a follow-up to this elog, double-counting of pixel defects in the production overview page was fixed in 3a2c6772.

261

Mon May 11 21:37:43 2020

Fixed the BB defects plots in the production overview page

0407e04c: attempting to fix the BB defects plots in the production overview page (seems mostly related to the 17 to 10 C change)
f2d554c5: it appears that BB2 defect maps were not processed correctly

262

Mon May 11 21:41:15 2020

17 to 10 C changes in the production overview page

0c513ab8: a few more updates on the main production overview page related to the 17 to 10 C change

265

Wed May 13 23:16:37 2020

Fixed double-counting of pixel defects in the production overview page

Dinko Ferencek wrote:

As a follow-up to this elog, double-counting of pixel defects in the production overview page was fixed in 3a2c6772.

A few extra adjustments were made in:

38eaa5d6: also removed double-counting of pixel defects in module maps in the production overview page
51aadbd7: adjusted the trimmed threshold selection to the L1 replacement conditions

274

Tue May 26 22:56:46 2020

FTs for ETHZ

Module list1

The following FullQualification tar file have been uploaded to CERNBox

rsync -avPSh --no-r --include="M1536_FullQualification_2020-04-29*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1537_FullQualification_2020-04-29*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1539_FullQualification_2020-05-11*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1540_FullQualification_2020-05-04*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1541_FullQualification_2020-04-29*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1542_FullQualification_2020-04-22*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1543_FullQualification_2020-04-29*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1545_FullQualification_2020-03-26*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1547_FullQualification_2020-04-29*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1548_FullQualification_2020-04-30*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1550_FullQualification_2020-04-30*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1551_FullQualification_2020-04-30*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1552_FullQualification_2020-04-30*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1553_FullQualification_2020-04-30*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1554_FullQualification_2020-04-22*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1565_FullQualification_2020-04-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1566_FullQualification_2020-04-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1568_FullQualification_2020-04-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1569_FullQualification_2020-04-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/

275

Tue May 26 23:00:50 2020

Problem with external disk filling up too quickly

The external hard disk (LaCie) used to back up the L1 replacement data completely filled up after transferring ~70 GB worth of data even though its capacity is 2 TB. The backup consists of copying all .tar files and the WebOutput/ subfolder from /home/l_tester/L1_DATA/ to /media/l_tester/LaCie/L1_DATA/ The corresponding rsync command is

rsync -avPSh --include="M????_*.tar" --include="WebOutput/***" --exclude="*" /home/l_tester/L1_DATA/ /media/l_tester/LaCie/L1_DATA/

It was discovered that /home/l_tester/L1_DATA/WebOutput/ was by mistake duplicated inside /media/l_tester/LaCie/L1_DATA/WebOutput/ However, this could still not explain the full disk.

The size of the tar files looked fine but /media/l_tester/LaCie/L1_DATA/WebOutput/ was 1.8 TB in size while /home/l_tester/L1_DATA/WebOutput/ was taking up only 50 GB and apart from the above-mentioned duplication, there was no other obvious duplication.

It turned out the file system on the external hard disk had a block size of 512 KB which is unusually large. This is typically set to 4 KB. In practice this meant that every file (and even folder), no matter how small, always occupied at least 512 KB on the disk. For example, I saw the following

l_tester@pc11366:~$ cd /media/l_tester/LaCie/
l_tester@pc11366:/media/l_tester/LaCie$ du -hs Warranty.pdf
512K Warranty.pdf
l_tester@pc11366:/media/l_tester/LaCie$ du -hs --apparent-size Warranty.pdf
94K Warranty.pdf

And in a case like ours, where there are a lot of subfolders and files, many of whom are small, a lot of disk space is effectively wasted.

The file system used on the external disk was exFAT. According to this page, the default block size (in the page they call it the cluster size) for the exFAT file system scales with the drive size and this is the likely reason why the size of 512 KB was used (however, 512 KB is still larger than the largest block size used by default). The main partition on the external disk was finally reformatted as follows

sudo mkfs.exfat -n LaCie -s 8 /dev/sdd2

which set the block size to 4 KB.

283

Thu Jun 4 11:32:46 2020

FTs for ETHZ

Module list2

The following FullQualification tar file have been uploaded to CERNBox

rsync -avPSh --no-r --include="M1538_FullQualification_2020-04-29*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1546_FullQualification_2020-04-02*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1549_FullQualification_2020-05-04*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1557_FullQualification_2020-04-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1570_FullQualification_2020-04-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1571_FullQualification_2020-05-04*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1572_FullQualification_2020-04-08*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1573_FullQualification_2020-04-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1574_FullQualification_2020-05-05*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1576_FullQualification_2020-04-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1577_FullQualification_2020-04-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1578_FullQualification_2020-04-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1579_FullQualification_2020-04-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1580_FullQualification_2020-04-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1581_FullQualification_2020-05-05*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1582_FullQualification_2020-05-11*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1583_FullQualification_2020-04-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1584_FullQualification_2020-04-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1585_FullQualification_2020-04-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1586_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1587_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1588_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1589_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1590_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1591_FullQualification_2020-04-03*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1592_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1593_FullQualification_2020-04-07*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1595_FullQualification_2020-05-06*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/

289

Tue Jun 16 00:45:48 2020

FTs for ETHZ

Module list 3

The following FullQualification tar file have been uploaded to CERNBox

rsync -avPSh --no-r --include="M1596_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1597_FullQualification_2020-04-30*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1598_FullQualification_2020-05-04*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1599_FullQualification_2020-03-25*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1600_FullQualification_2020-04-28*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1601_FullQualification_2020-03-18*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1602_FullQualification_2020-03-18*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1603_FullQualification_2020-03-18*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1604_FullQualification_2020-03-18*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1605_FullQualification_2020-03-19*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1606_FullQualification_2020-05-11*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1607_FullQualification_2020-03-19*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1608_FullQualification_2020-03-19*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1609_FullQualification_2020-03-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1610_FullQualification_2020-03-20*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1611_FullQualification_2020-03-20*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1612_FullQualification_2020-03-20*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1613_FullQualification_2020-03-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1614_FullQualification_2020-03-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1615_FullQualification_2020-03-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1618_FullQualification_2020-03-23*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1619_FullQualification_2020-03-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1620_FullQualification_2020-03-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1622_FullQualification_2020-03-24*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1623_FullQualification_2020-04-15*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1624_FullQualification_2020-03-25*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1626_FullQualification_2020-03-25*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1627_FullQualification_2020-03-26*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1628_FullQualification_2020-03-26*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1629_FullQualification_2020-03-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1631_FullQualification_2020-03-27*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1641_FullQualification_2020-04-01*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1642_FullQualification_2020-04-01*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1643_FullQualification_2020-04-01*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1644_FullQualification_2020-04-01*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1645_FullQualification_2020-04-02*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1647_FullQualification_2020-04-02*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1648_FullQualification_2020-04-02*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1649_FullQualification_2020-05-05*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1651_FullQualification_2020-04-03*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1653_FullQualification_2020-04-22*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1654_FullQualification_2020-04-09*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1655_FullQualification_2020-04-06*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1656_FullQualification_2020-04-06*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1657_FullQualification_2020-04-15*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1658_FullQualification_2020-04-07*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1659_FullQualification_2020-05-06*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1660_FullQualification_2020-05-05*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1661_FullQualification_2020-04-08*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1662_FullQualification_2020-04-16*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1663_FullQualification_2020-04-08*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1664_FullQualification_2020-04-08*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1665_FullQualification_2020-04-09*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1666_FullQualification_2020-04-09*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1667_FullQualification_2020-05-05*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1668_FullQualification_2020-05-05*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1669_FullQualification_2020-04-14*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1670_FullQualification_2020-04-14*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1672_FullQualification_2020-04-14*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1673_FullQualification_2020-04-15*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1674_FullQualification_2020-04-15*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1675_FullQualification_2020-04-16*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/
rsync -avPSh --no-r --include="M1676_FullQualification_2020-04-16*.tar" --exclude="*" /home/l_tester/L1_DATA/* /home/l_tester/DATA/L1_DATA_Backup/CERNBox_Dropbox/

293

Sun Jul 5 13:20:55 2020

Disk cleanup on the lab PC (pc11366.psi.ch)

Home folder was cleaned up by deleting old pXar output .log and .root files. The full list is in the attached file. This released around 11 GB of disk space.

In addition, the following MoReWeb output folders which are no longer needed were deleted

/home/l_tester/L1_DATA/WebOutput/MoReWeb/FinalResults/REV001/R001/M1549_FullQualification_2020-02-04_23h22m_1580854955/
/home/l_tester/L1_DATA/WebOutput/MoReWeb/FinalResults/REV001/R001/M1595_FullQualification_2020-03-12_11h05m_1584007524/
/home/l_tester/L1_DATA/WebOutput/MoReWeb/FinalResults/REV001/R001/M1598_FullQualification_2020-03-12_11h05m_1584007524/
/home/l_tester/L1_DATA/WebOutput/MoReWeb/FinalResults/REV001/R001/M1646_FullQualification_2020-04-02_08h41m_1585809682/

This released around 805 MB of disk space.

302

Sat Sep 12 23:45:56 2020

pXar parameter files were converted to POS configuration files by executing the following commands on the lab PC at PSI

Step 1 (needs to be done only once, should be repeated only if there are changes in modules and/or their locations)
cd /home/l_tester/L1_SW/MoReWeb/scripts/
python queryModuleLocation.py -o module_locations.txt -f
Next, check that /home/l_tester/L1_DATA/POS_files/Folder_links/ is empty. If not, delete any folder links contained in it and run the following command
python prepareDataForPOS.py -i module_locations.txt -p /home/l_tester/L1_DATA/ -m /home/l_tester/L1_DATA/WebOutput/MoReWeb/FinalResults/REV001/R001/ -l /home/l_tester/L1_DATA/POS_files/Folder_links/

Step 2
cd /home/l_tester/L1_SW/pxar2POS/
for i in `cat /home/l_tester/L1_SW/MoReWeb/scripts/module_locations.txt | awk '{print $1}'`; do ./pxar2POS.py -m $i -T 50 -o /home/l_tester/L1_DATA/POS_files/Configuration_files/ -s /home/l_tester/L1_DATA/POS_files/Folder_links/ -p /home/l_tester/L1_SW/MoReWeb/scripts/module_locations.txt; done

The POS configuration files are located in /home/l_tester/L1_DATA/POS_files/Configuration_files/

306

Tue Nov 10 00:50:47 2020

danek kotlinski wrote:

Dinko Ferencek wrote:

Hi Danek,

I think I implemented everything that was missing. The full list of code updates is here.

Best,
Dinko

308

Tue Jan 19 15:12:12 2021

M1560 in position bpi_sec1_lyr1_ldr1_mod3 was replaced by M1613.

The POS configuration files were re-generated and placed in /home/l_tester/L1_DATA/POS_files/Configuration_files/.

The old version of the files was moved to /home/l_tester/L1_DATA/POS_files/Configuration_files_20201110/.

309

Mon Jan 25 13:03:22 2021

The output POS configuration files has '_Bpix_' instead of '_BPix_' in their names. The culprit was identified to be the C3 cell in the 'POS' sheet of Module_bookkeeping-L1_2020 Google spreadsheet which contained 'Bpix' instead of 'BPix' which was messing up the file names. This has been fixed now and the configuration files regenerated.

The WBC values were also changed to 164 for all modules using the following commands

cd /home/l_tester/L1_SW/pxar2POS/
./pxar2POS.py --do "dac:set:WBC:164" -o /home/l_tester/L1_DATA/POS_files/Configuration_files/ -i 1

This created a new set of configuration files with ID 2 in /home/l_tester/L1_DATA/POS_files/Configuration_files/.

The WBC values stored in ID 1 were taken from the pXar dacParameters*_C*.dat files and the above procedure makes a copy of the ID 1 files and overwrites the WBC values.

123

Sun Mar 22 13:57:25 2020

M1617 failed

Andrey Starodumov wrote:

ROC8 of M1617 is programmable but no readout from it.
Silvan noticed that a corner of one ROC of this module is broken,
this is exactly ROC8.

Interesting that the phase finding works fine, the width of the valid region is 4, so quite
good. ROC8 idneed does not give any hits but the token passed through it, so the overall
readout works fine. There re no readout errors.
The crack on the corner of this ROC is clearly visible.
I wonder how this module passed the tests in Helsinki?

124

Sun Mar 22 14:02:20 2020

M1616 failed

Andrey Starodumov wrote:

ROC10 of M1616 is not programmable, no readout from ROC10 and ROC11.
Visual inspection of wire bonds - OK
To module doctor!

For me ROC10 is programmable.
It looks like there is not token pass through ROC11.
This affects the readout of ROCs 11 & 10.
Findphases fails because of the missing ROC10&11 readout.

125

Sun Mar 22 14:05:20 2020

M1615 failed

Andrey Starodumov wrote:

M1615 is programmable but "no working phases found"
Visual inspection of wire bonds - OK.
To module doctor!

For me this module is working fine.
I could run phasefinding and obtained a perfect PixelAlive.
I left this module connected in the blue-box in order to run more advanced tests from home.

156

Mon Mar 30 14:52:59 2020

FT of M1629, M1630, M1631, M1632

I have tested M1630.
I see not problem with ROC1. See the attached plot.
There is one dcol in ROC12 which shoes the "pattern" problem seen in a few other ROCs.
I think this module is fine, should be B. Could be retested.

Mon Feb 3 11:28:38 2020

Anrey Starodumov

4 modules FT: 1536, 1537,1538, 1540

Jan 28: Reception test for 1536, 1537, 1538 OK
Then cap gluing
Jan 29
- Reception test for 1540 OK
- FT:
-- 1536: B/B/B (-20/-20/+10). Electrical Test is B due to mean noise >200e. IV: 3umA for -20 and +10, slope >300
-- 1537: C/B/B. First -20 is C due ROC10 >200 TrimBits failure, noise >200e. IV: A
-- Trimbit test fails but Trimbit distribution is OK and Threshold reasonable (attachment)
==> Manually regraded to B
-- 1538: B/B/B (-20/-20/+10). Electrical Test is B due to mean noise >200e. IV: <0.2umA for +10, slope ~1
-- 1540: B/B/B (-20/-20/+10). Electrical Test is B due to mean noise >200e. IV: <0.8umA for +10, slope ~2.7

Mon Feb 3 15:35:09 2020

Anrey Starodumov