Every so often, I get reminded that I’m old, and I’ve been programming for almost 60 years, which is a long time. But 60 years in the business means I’ve seen a lot of things that young naïve programmers have never seen.
This comes up often when people talk about DOGE and the Wizards Academy Musk has put together to help investigate fraud, abuse, and, probably most of all, bureaucratic stupidity.
One of the things I see people — technical people, but young — saying about things like Social Security and the IRS is things like “just dump the whole database into Hadoop.”
The problem with that starts with the fact that it’s not in a database. It’s a wildly heterogeneous collection of different databases, ISAM files, and card images, and I would bet money that a lot of it is on old 7-track tapes. Some of these are probably stored in Iron Mountain or a similar installation. Also, some of the data may still be just on paper, as, apparently, government retirement records are.
So what Big Balls and the other wizards are going to need to do to start with is find the data.
I’m willing to bet there’s no single catalog of all the data sets. Having found the data, much of it is card images that almost certainly are only documented by COBOL copybooks. (Back in 2020, I wrote about COBOL for the Stack Overflow Blog when it suddenly became trendy as some of these systems desperately needed to be maintained.)
Hellbound and Down 