ASTRID Error Fix
Status:
This documentation applies to version 1.47.745. (7. May -2024) of the VdAstrid/CVdAstErrorFix device.
Contents:
- CVdAstErrorFix overview
- Error Fix device
- ASTRID2 error fix scheduler device
- ASTRID automation device
- VdAstrid
- Microtron Modulators Automation
- F0: K100 turning off 'randomly'
- F8: K100 going to standby 'randomly'
- F1: Microtron Sector valve VVS61 and VVS62 Closed
- F2: Microtron VVR61 Roughing valve closed check
- F2: F3-F7: BMH11, QMH11, QMH21,QMV11,QMV21 supply magnet failure
- F9: ASTRID SMH11 failure
- F10: Timing CRIO communication check
- F11: Microtron BMH606 Dipole spike detection/recovery
- F12: InjBl quadrupole error recovery
CVdAstErrorFix overview
Device design notes - see document(s) in folder
O:\Nat_ISA-Ast2Doc\machine\Documentation\Controls\Automation\ErrorFix
Microtron Modulators Automation
The Microtron Modulator Error fix methods must only run if the Microtron bunker is closed and has been manual approved to run by operator after bunker closure.
A specific handling of this has been implemented in the device and is controlled by the cluster above.
This device will monitor the safety PLC, PSI60SFTY bunker closed status. If the bunker is opened, microtron modulator automation/error fix procedures should no longer be allowed to run. When blocked,
MicModAuto.automation is FALSE/NotAllowed =>
F0 K100 turning off 'randomly' and
F8: K100 going to standby 'randomly'
can not be trigged
Status:
MicModAuto.condOk - True when conditions for automation is fullfilled (bunker is closed)
MicModAuto.automation - status (Allowed/NotAllowed) maintained by this device
Control:
MicModAuto.setDisable - 'Push' button, Set Mic Modulation allowed to FALSE
MicModAuto.setEnable - 'Push' button, set by operator to enable automation after bunker closed
Source parameters used to monitor bunker closed:
PSI60SFTYmic.ClosedAccess (Must be 'HallClosed')
PSI00SFTYast.uTronD2 (Must be closed)
Remark: MicModAuto.automation going to 'Allowed' trigs automation A0: Microtron Bunker Closed
F0: K100 turning off 'randomly'
EFX00K100ast
Trigger
MachineMode: onAutoRecovery and
Microtron Modulator Automation on and
K100OPmic.stateTargetSet is off and K100OPmic.enableStateTargetOff has not been activated (i.e. a non - intended K100 off).
Action
Set Decaying
Wait up to 30 minutes for K100 active (Communication ok)
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Request remote
Reset
Set K100 to Standby
Now wait 30 minutes for stateRead to reach Standby
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Record value of MicRFPulse.FwdRF
Reduce CRE61OSC.RFLevel by 2 dBm
Set K100 to Trig
Set A2 beammode to Injection Adjust
Slowly increase CRE61OSC.RFLevel by 0.05 dBm until MicRFPulse.FwdRF is equal to recorded level
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Trig a MicAutoAlign(send true to AutoAlignAstInj.AlignNow) and wait until MicAutoAlign is done
If Ok, Restore BeamMode (Filling)
Set MicAutoAlign to run at an interval of 2 h
Action Progress:
0: Inactive
1: Trigged K100 Turn of recovery
2: Run 'Turn of recovery' started:
10000: Detected K100 not active (no communication)
1xxxx: Waiting for K100 active, counting up from 10000. *1
101: K100 active/commicating
102: K100 not active/commicating
103: not ok after K100 active check
104: Machine Mode no longer in OnAutoRecovery (after initial wait)
200: Proceding ok, continue actions
20000: K100 Reset and set to Standby
2xxxx: Waiting for Standby, xxxx seconds. *1
300: Standby ok
301: Standby not reached
302: Machine Mode no longer in OnAutoRecovery (after waiting for standby)
40000: Start adjusting RF forward power
4xxxx: Adjusting RF forward power, steps. *1
400: Machine Mode no longer in OnAutoRecovery (after adjusting RF power)
50000: Starting auto align
5xxxx: Waiting for auto align to finish. *1
600: Setting MicAutoAlign to run at an interval of 2 h
F8: K100 going to standby 'randomly'
EFX00K100STBYast
Trigger
MachineMode: onAutoRecovery and
Microtron Modulator Automation on and
(stateTargetSet going to standby without K100OPmic.enableStateTargetOff enabled) or
(interlocks exists)
Action
Check known to go wrong interlock conditions, Reset K100, Set Trig mode, set Ring operation Topup mode:
Set Decaying
Wait up to 30 minutes for K100 active (Communication ok)
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Reset K100
Wait for CCPS.PS1 < 100 V
Check that MachineMode is still in OnAutoRecovery - if not so, abort action.
Set Trig mode
Check recover state - and retry recovery if not ok.
If Recovered, Restore BeamMode (Filling)
Action Progress:
0: Inactive
1: Trigged
2: Checking active status
10000: Detected K100 not active (no communication)
1xxxx: Waiting for K100 active, counting up from 10000. *1
101: K100 active/commicating
102: K100 not active/commicating
103: not ok after K100 active check
104: Machine Mode no longer in OnAutoRecovery (after initial wait)
Recovering, recovery try number 'n'
n000: Waiting for next recover try
n101: Machine Mode no longer in OnAutoRecovery (next revery try)
n100: Resetting K100
n2xx: Waiting for CCPS voltage is below 100 V, xx seconds. *1
n300: Setting K100 trig
n400: Checking for recovered
n500: Machine Mode no longer in OnAutoRecovery (while checking for recovering)
5000: Recovered succesfully
6000: Recovered failed, recovery has been run
7000: Recovered failed, recovery has not been run
F1: Microtron Sector valve VVS61 and VVS62 Closed
EFX00MICVVSast
Trigger
VVS61 Closed or VVS62 closed while MachineMode at least in OnBasicAuto and StatusAst2.BeamMode not in 'Injection Adjust'. The trigger is NOT activated if VVS61ast.close or VVS62ast.close is pressed shortly before valve close status detected.
Action
Set Decaying
Retry recovery up to 'fixRetries' times
Wait up to 3 minutes for pressures to be OK
Check that MachineMode is still at least in OnBasicAuto - if not so, abort.
If 'Ok to open'
Try opening the valves
Test if valves reopened
End If 'Ok to open'
Check that MachineMode is still at least in OnBasicAuto - if not so, abort further retries.
End Retry recovery
Action Progress:
0: Inactive
1: Trigged
2: Set daying
Recovering, recovery try number 'n'
nxxx: Wait for pressures to be OK, waited xxx seconds (xxx 0-180). *1
n200: Check status
n300: Reopen valves
n400: Test if valves has reopened
n500: Opened failed
n800: Presure not ok to open
n900,n901: Machine Mode no longer at least in OnBasicAuto
3: Not recovered
4: Recovered
F2: Microtron VVR61 Roughing valve closed check
EFX00VVR61ast
Trigger
VVR61 Roughing valve closed. If the manuel closed (user pressed VVR61ast.close shortly before closed) the action is NOT trigged.
Action
Set Decaying
Retry recovery up to 'fixRetries' times
Wait up to 30 seconds.
'Ok to open' = VGP63 pressure ok and VPT61 running
If 'Ok to open'
Try opening the valve
Test if valve reopened
End If 'Ok to open'
End Retry recovery
Action Progress:
0: Inactive
1: Trigged
2: Set daying
Recovering, recovery try number 'n'
n100: Wait a 5 (first try)/30(following tries) seconds
n500: Open valve
n600: Not opened
3: Not recovered
4: Recovered
F3-F7: BMH11, QMH11, QMH21,QMV11,QMV21 supply magnet failure
EFX00BMH11ast, EFX00QMH11ast, EFX00QMH21ast, EFX00QMV11ast, EFX00QMV21ast
Trigger
If StatusAst2.MachineMode is 'onAutoRecovery' and xxxx.Power is 'On':
Trig if either a supply, phase, supply water, supply temp,earth, main or filter error is detected.
Action
Store Astrid2 operation mode
Set Decaying
Retry recovery up to 'fixRetries' times
Wait for next try
Check that MachineMode is still in OnAutoRecovery - if not so, abort.
Reset Error
Check that MachineMode is still in OnAutoRecovery - if not so, abort.
If 'Ok'
Turn supply off
Wait 2000 ms
Turn supply on
Wait 3000 ms
End If 'Ok'
Check that MachineMode is still in OnAutoRecovery
End Retry recovery
If Ok: Restore Astrid2 operation mode
Action Progress:
0: Inactive
1: Trigged
2: Recovering started, Set Decaying
Recovering, recovery try number 'n'
n001: Waiting for next try
n002: Reset supply and wait
n003: Try to restart supply
n004: Check to see if recovered
n010: Errors still present
n900,n901,902: Machine Mode no longer in OnAutoRecovery
F9: ASTRID SMH11 failure
EFX00SMH11ast
Trigger
Machine mode is 'onAutoRecovery' and SMH11ISP.Power is 'on':
MPS,overload,reg,overcurrent,earth,magnet_water,phase or breaker error is present
Action
Store Astrid2 operation mode
Set Decaying
Retry recovery up to 'fixRetries' times
Check that MachineMode is still in OnAutoRecovery - if not so, abort.
Reset Error
Check that MachineMode is still in OnAutoRecovery - if not so, abort.
If 'Ok'
Turn supply off
Wait 1 second
Turn supply on
End If 'Ok'
Check that MachineMode is still in OnAutoRecovery
End Retry recovery
If Ok: Restore Astrid2 beam mode
Action Progress:
0: Inactive
1: Trigged
2: Recovering started, Set Decaying
Recovering, recovery try number 'n'
n001: Waiting for next try
n002: Reset supply and wait
n003: Try to restart supply
n010: Errors still present
n900,n901,902: Machine Mode no longer in OnAutoRecovery
F10: Timing CRIO communication check
EFX00CRIOast
Trigger
CRIO98 Communication status changed
Action
If communication lost, set Beam Mode decaying
If communication reetablished and decaying mode prev set by communication lost, set prev. beam mode again.
F11: Microtron BMH606 Dipole spike detection/recovery
EFX00BMH606SPKast
Trigger
Machine mode is 'OnAutoRecovery' and
ASTRID automation 'aut2 BMH606IPS Wash' is not running and
BMH606IPSast.Main is 'On' and
Iw has not been changed and
Trig on spike in BMH606IPSast.Ir detection:
A spike is defined as:
(Ir < (Iw - 0.30 A)) OR (Ir > (Iw + 0.10 A)
Action
Set Decaying
Wait 10 seconds
Trigger a BMH606 wash
Wait for 'Aut2: BMH606IPS Wash' finished
If Ok: Restore Astrid2 beam mode
Action Progress:
0: Inactive
1: Trigged
2: Recovering started, Set Decaying
3: BMH606 Wash trigger send
1xxx: Waiting for wash start, xxx seconds
4: Wash in progress detected
2xxx: Waiting for wash finished, xxx seconds
5: Wash finished ok
101: BMH606 wash not finished in time
102: Wash was not started by trig wash
F12: InjBl quadrupole error recovery
EFX00INJBLQUADast
Trigger
Machine mode is 'OnAutoRecovery' and
One of the following is below 0.2 A
QMH63IQP.adc
QMH64IQP.adc
QMV63IQP.adc
QMV64IQP.adc
Action
Test that ConSys connections are OK (no network failure)
If ConSys problems, wait for ConSys recovery
Set Decaying (new from 3/10-23)
Send true to QMx6xREL.reset
Wait 5 s
If one of the parameters is still below 0.2 A then GiveUp
If Recovered: Restore Astrid2 beam mode(new from 3/10-23)
Action Progress:
0: Inactive
1: Trigged
2: Recovering started
1xxx: Waiting for ConSys connections all connected, xxx seconds*1
15xx: Waiting for ConSys feature sources valid, xxx seconds*1
3: QMx6xREL Reset
10x: Wait 5 seconds (x counting down)
2xxx: Check all qmx for recovered (above 0.2 A), xxx seconds*1
5: Recovered ok
500: Action aborted/stopped
501: ConSys connections not ok
502: Inj Bl quad recovery failed (still at least one below 0.2 A)
*1
ActionProgress: When counting up, only the first and the last value are added to the read/write histories in order to make the histories more readable.
Last Modified 13 May 2024