Friday, March 24, 2006

Mainframe job skills: historical survey

From the 1960s until well into the 1990s there was a well-established large scale computing culture in large companies, particularly financial institutions, life and health insurers and government agencies, centered around large mainframes and, at first, batch processing followed quickly by “demand” and “real time” – effectively online. At one time there were five major companies, dominated by IBM, and I tried to work for a competitor, Sperry Univac.

IBM beat out all of the competition by sheer size and a certain kind of disciplined professionalism. The job control language (JCL) developed a reputation with some people as verbose and unwieldy, but knowledge of JCL was an essential skill by the 1970s. In fact, at one time there was DOS, which was quickly overtaken my MVS. Many shops in the 1970s developed their own commercial applications in house, largely with COBOL and sometimes assembler; the applications would start out with daily and end-of-month batch cycles and followup with online access through TSO and then CICS. Mainframe databases – hierarchal like IMS or relational like DB2, and sometimes from other vendors (like IDMS, Adabas, or Datacom DB) became popular. CICS was indeed the main teleprocessing monitor, although IMS had its DC and there was Datacomm DC. By the early 1980s, most companies provided individual CRT terminals for employees to do their work “online.” Gradually, inhouse-written applications were replaced by packages from major mainframe software vendors.

Procedural programming rapidly became more efficient in the 1970s, as structured programming technologies and top-down testing techniques became published, and as so-called "fourth generation languages" like DYL280 and Easytrieve came into populartity. One such package, SAS, with its unique philosophy of data and procedure steps, became a comprehensive application develop system of its own, and is particularly popular in analyzing data for statistical patterns, as would be needed for public policy research in areas like health care.

Gradually, the job culture came to reward those programmers who developed skills in the mainstream IBM products, as well as major packages from a variety of closely related vendors like CA (for job scheduling and source management).

Security started to become a major issue in the 1980s. By around 1990, most major companies did not allow programmers to update production files without specific access, although there were many loopholes. The integrity of source management was understood as a significant security issue by the early 1990s.

Even as personal computers and the Internet became important, the job market in the mainframe area remained very strong in the 1990s because of the need for Y2K conversions. The year 2000 came with very few actual problems.

After the Y2K experience, it seems that many companies transferred much of their mainframe maintenance overseas. This even included nightcall support, which could be accomplished from India during their day shift. It seemed that for a while that the mainframe job market was almost vanishing. Yet, demand remained strong and paid top dollar in specialized areas where detailed technical expertise was required. These areas included particularly DB2, Vantage (for insurance and annuities). some of the Case tools, and sometimes, surprisingly, IMS, as well as data warehousing. Analysts with enough experience in specific business areas like Medicaid MMIS could get jobs.

There seemed to be a curious phenomenon here. Conventional wisdom in the 1980s and 1990s had suggested that the computer professional spread out into many areas, but after 2000 the market was rewarding extreme specialization. Companies would scour the country looking for specific skill sets in areas that are perceived as becoming obsolete, like IMS. There was a reward for staying with old skill sets, which was not what many people had expected.

There has been a lot of speculation as to whether the mainframe job market that we knew in the past will come back, whether companies will pull back operations that they offshored. Recruiters tell me that they do not, as of early 2006, see much evidence that this has happened. If it did happen, there would be an increase in demand for older professionals in their 50s and 60s.

No comments: