05 · Processes
The batch window was a tool. It wasn't a requirement.
Mainframes do work overnight partly because that's when capacity was free. Before you specify a job, separate what the business needs from what the machine once required. Then do the same for messages and partner files.
JCL and batch · at Tidewater
The nightly sweep becomes one timer per payment.
ACLP210 scans every due payment each night. What the business needs is follow-up when a payment is late. The Standard already has the answer: a process with a timer, which wakes up and re-reads what's true then (DATA-021).
//ACLP210 JOB (TW),'LATE SWEEP',CLASS=B
//STEP010 EXEC PGM=SORT
//SORTIN DD DSN=TW.PROD.PMTDUE(0),DISP=SHR
//SORTOUT DD DSN=&&PMTSRT,DISP=(NEW,PASS)
//SYSIN DD *
SORT FIELDS=(1,4,CH,A,18,8,CH,A)
INCLUDE COND=(17,1,CH,NE,C'G')
/*
//STEP020 EXEC PGM=ACLP210,COND=(4,LT,STEP010)
//PMTIN DD DSN=&&PMTSRT,DISP=(OLD,DELETE)
//FEEOUT DD DSN=TW.PROD.LATEFEE(+1),
// DISP=(NEW,CATLG,DELETE)
# Late Payment Follow-up Code: LPF Application: payments ## Parameters | Parameter | Type | Meaning | |------------|---------|------------------------------| | grace days | number | Days after the due date | | fee waived | boolean | The bank charges no late fee | ## Stages 1. Wait out the grace days (automated, timer) 2. Check payment and loan (automated) 3. Assess fee (automated) 4. Post the fee (sub-process: post-late-fee) 5. Contact bank (human, skill: late follow-up) ## Rules - LPF-001: Follow-up starts only if the scheduled payment is still `expected` once the grace days have passed. - LPF-002: Follow-up acts only on a loan that is still active when the process wakes. - LPF-005: A government-guaranteed loan never enters late payment follow-up.
An excerpt, with five stages, the Standard's usual machine size.
INCLUDE COND removes class G before the program ever runs. The COBOL never mentions guaranteed loans, so reading it alone gives you the wrong rule set.
The scheduler runs ACLP210 after TWPOST and TWBKUP. Posting matters, because a payment posted tonight isn't late. The backup doesn't affect the result, so that dependency is dropped.
TWSTMT writes every borrower's statement by the fifth business day. That really is bulk, so it becomes a scheduled process below.
KeelTrack's KTNIGHT CL program submits KTLAPSE with SBMJOB, from a WRKJOBSCDE entry. Operators watch it on WRKACTJOB and restart it from a handwritten note. That note is evidence, and it describes what an idempotent stage must do.
True bulk · at Tidewater
A schedule starts it. A process runs it.
Some jobs really are bulk. An EventBridge schedule starts a documented process, and Step Functions runs the batch in chunks and handles success and failure. What happens when part of a run fails is a rule the business decides.
# Monthly Statements Code: STM Application: payments ## Stages 1. Check the calendar (automated) 2. List the loans (automated) 3. Produce statements (sub-process, in chunks) 4. Review failures (human, skill: reconciliation) 5. Close the run (automated) ## Who can start it A schedule: every day at 6:00 a.m. Central. ## Flow | From | When | To | |--------------------|----------------------------|------------------| | Check the calendar | not the fifth business day | The request ends | | Check the calendar | it is | List the loans | | Produce statements | any failed | Review failures | | Produce statements | none failed | Close the run | | Review failures | resolved | Close the run | ## Timeouts | Stage | Overall | If not done | |--------------------|---------|-----------------------------| | Produce statements | 2 days | continue at Review failures | | Review failures | 2 days | abort | ## Volume | Items | Expected per run | Decrypts per item | At once | |------------|------------------|-------------------|---------| | statements | 19,000 | 1 | 50 | ## Rules Direct - STM-001: Every active loan gets one statement for each month, by the fifth business day. - STM-002: A loan that closed during the month gets a final statement. Indirect - STM-003: A statement that keeps failing goes to a person, and the rest of the run continues. - STM-004: If more than 5% of statements fail, the run stops and goes to a person. - STM-005: No borrower gets two statements for the same month.
A schedule can't express "the fifth business day" or a bank holiday. It fires every day, and the first stage decides, so the rule sits in the spec instead of in a scheduler's definition.
Reading 19,000 balances looks like bulk extraction. The Volume table gives this run alone an allowance, and the permit review sees it. Going over still raises the alarm.
Each chunk builds its PDFs with a small piece of custom code, and publishes them to the bank's documents. No reporting product is involved, because a statement is a document produced on a schedule.
Statements show balances, which stay on Anchor Core. So TWSTMT moves only once Anchor Core supplies a period-end extract, which comes in through quarantine like any other file.
IBM MQ · at Tidewater
Pick the pattern by what the exchange needs.
A queue manager doesn't translate to a service. Each flow gets its own answer from the Standard's integration list, and three of Tidewater's flows came out three different ways.
| Queue | What it really does | Trap | Becomes |
|---|---|---|---|
TW.FUNDING.NOTICE.IN | Carries a clearing service's funding notices into Anchor Core | The consumer assumes the remittance file came first. The 3% that didn't are parked for manual work. | SQS with a DLQ on the source queue. Reconciliation already accepts either order, so the pile disappears. |
TW.LOAN.STATUS.PUB | Publishes loan changes to BankLink, KeelTrack and the warehouse | Every message holds the whole record, balance included | The outbox relayed to EventBridge. Events carry identifiers, and each consumer reads what it's entitled to. |
TW.DEAD | The dead-letter queue | 2,214 messages, some from 2019, and nobody watches it | A DLQ with an alarm, an owner and a redrive runbook. Each old message gets an answer before cutover. |
On IBM i, data queues (DTAQ) deserve the same treatment. They're easy to miss because they don't look like messaging infrastructure.
Postings to a retained system · at Tidewater
What happens when it fails is a business decision.
The ledger stays on Anchor Core, so the fee has to reach it. Posting it is a documented process, with direct rules about the posting and indirect rules about what happens when a system fails. Outcomes are flow and waits are timeouts, and Step Functions runs both with no additional code.
# Post Late Fee Code: PLF Application: payments ## Stages 1. Post fee (automated, posts to Anchor Core) 2. Wait for confirmation (automated) 3. Check with Anchor Core (automated) 4. Review (human, skill: reconciliation) 5. Record outcome (automated) ## Flow | From | When | To | |------------------------|---------------------------|------------------------| | Post fee | sent | Wait for confirmation | | Wait for confirmation | accepted | Record outcome | | Wait for confirmation | refused: not enough funds | Record outcome | | Wait for confirmation | refused: any other reason | Review | | Check with Anchor Core | found | Record outcome | | Check with Anchor Core | not found or no answer | Review | | Review | resolved | Record outcome | ## Timeouts | Stage | Overall | If not done | |------------------------|---------|-----------------------------------| | Wait for confirmation | 1 day | continue at Check with Anchor Core | | Check with Anchor Core | 4 hours | continue at Review | | Review | 3 days | abort | ## Rules Direct - PLF-001: A fee on an account without enough funds is recorded as owed, not dropped. - PLF-002: A fee refused for any other reason goes to a person, with the reason. Indirect - PLF-003: A fee doesn't show as posted until Anchor Core confirms it. - PLF-004: A fee with no confirmation after one day is checked before anything else happens. - PLF-005: A fee is never posted twice. ## If aborted The fee stays unposted, and the bank's late follow-up shows that it needs attention.
A timeout says when to stop waiting, not whether the fee landed. Retrying automatically could charge the borrower twice. Anchor Core can't recognize a duplicate, so the process checks instead, which needs a small new inquiry on Anchor Core.
The fee goes by MQ into the nightly TWPOST run, and the confirmation arrives in the next morning's file. Only that file's receiver and the posting step are hand-written. The one-day wait is one row in the Timeouts table.
A reversal is a new posting that references the original. Daily reconciliation of fee counts and totals per bank is a process with a timer, with rules of its own.
Partner files · at Tidewater
Silent repairs become visible errors.
Harborside sends Standard Remittance files by Connect:Direct, and Pinecrest uploads hand-prepared fixed-width files through BankLink. TWRMT100 writes both straight into VSAM, and it quietly drops Pinecrest's zero-amount rows. No bank has ever been told.
Before
- Connect:Direct or BankLink delivers the file
TWRMT100writes rows directly to VSAM- Bad rows vanish without a report
- The trailer's control total is checked by the operator
After
- Harborside's system signs in by OIDC and uploads to quarantine; Pinecrest's staff sign in and upload by hand
- One ingestion sub-process per format, nothing in a domain until commit
- Row-level errors go back to the bank, a fixed defect
- RMT-001: A file's rows must add up to the total its trailer states.
Rules
MIG-018 to MIG-021, MIG-040, MIG-042
- MIG-018
A batch job is specified by its business purpose and its deadline, not by its sequence of steps.
- MIG-019
An ordering between jobs is kept only where a data dependency is shown.
- MIG-020
Restart and rerun procedures become idempotent stages, with progress visible while they run.
- MIG-021
Every message flow uses one pattern from the Standard's integration list, or a recorded design decision.
- MIG-040
Every posting to a retained system is a documented process stating what the business does when the target accepts, refuses, or does not answer.
- MIG-042
A job that stays bulk becomes a scheduled process that runs in chunks and declares its volume, with its business calendar in the first stage.