Tuesday, April 10, 2012

My history with crude data warehousing; repairing damaged CD's

One of the first calls I had after my end-2001 layoff was for a “data warehousing” contract in downtown Minneapolis at Wells Fargo.  

I was just starting the job search, was still in outplacement, and had no real concept of how picky clients would be at finding matching candidates – a practice which meant it was hard to break into any other area of expertise.

A site that explains the concept is LGI’s, here.

But I had indeed encountered some rather crude experience with the concept.  In 1976, I wrote a “tape save” in COBOL (back on the old Univac 1110) for NBC of the general ledger data. 

Then in the early 1990s, we effectively had a manual data warehouse of all of our salary deduction bills – microfiche.  Rather like the way you used to have to look at newspapers in public libraries.  (It was always hard to remember how to use those machines.) 

The biggest danger in those days is that no one tried to use the data, so if you didn’t force yourself to look at it (and read it back somehow) as a programmer, there was a risk that a “warehouse” could become unusable five years later when someone needed it.  Preventing such a possibility was part of a programmer’s “work habits”.

For Y2K we “warehoused” our tests manually, in cardboard moving boxes, shipped to an iron mountain.
And for a disaster recovery exercise in March 1999, we captured entire cycles of data on GDG’s.  It was all rather a crude exercise.

So I really didn’t have the specialized experience one needed for that 2002 job.

Here’s something else. There is a “WikiHow” on fixing scratched CD’s, which many small businesses use for their own local data retention (since they’re optical, they can’t be harmed by magnetic attacks or accidents).  I find this link amazing. Here it is.

Here's another little tip.  If you have a lot of classical CD sets, make sure you throw away any foam separating the discs.  Over twenty years, the foam can make a mess. 

Wikipedia attribution link for commercial data warehousing diagram. 

No comments: