Wednesday, August 03, 2011

Budget simulation: It's hard to prove that the numbers are right!

The recent debt ceiling fiasco, especially inasmuch as it required a number of simulation model calculations from the Congressional Budget Office (CBO), calls to mind one particular issue: the importance of the reliability of the numbers that it publishes: that is, the correctness of the mathematical model used and of the translation of the specifications of that model into applications programming code, using the right data elements and typing them correctly.

I may have reported this matter before, but in January 1989, while I was working as a COBOL programmer for a small Blue Cross/Blue Shield subsidiary called “Consolidated Consulting Group” (CCG) in Washington, we had a crisis where a major client did not believe our numbers. I had kept very detailed paper records of all our simulations. Fortunately, I had access to Propac’s computer code and found it did not match the formulas printed in the Federal Register, with one data element (on a Medicare-related policy file) being different. 

When I changed to use the data element actually found in Propac’s code, we got the same numbers as Propac and the client believed us. I may have saved the business, which later that year got sold to Lewin/ICF.

No comments: