Project Manager
Home

 

 

DATA WAREHOUSE. PROJECT Mgr.        

Working for my employer INDEPENDENT TECHNOLOGIES  I developed for our main Client (SPRINT) a major Oracle application, written in Visual C++, under Windows NT and with ODBC connectivity.  The program -called C&C® (Contracts and Commissions)- calculates commissions due on 60,000 public phones, and administers the 50,000 contracts of those phones.  It can be easily adapted to work with any other RDBMS engine.  To deploy the system I was relocated to Kansas, where –for about two years- I had an office at the Client’s headquarters (in Overland Park).

 PREAMBLE: DATABASE EXPERTISE, CHRONOLOGY (Last two decades) .-

At the beginning of that era when microcomputers were introduced, I founded "Scientific Software" (Sept. 1978, Suffolk County, NY).  The first applications I developed were computer graphics programs.  Two years later, I resigned from British Airways were I had been a Telecommunications Officer for four years, sold my house on Long Island, NY, and moved to Puerto Rico where I dedicated full time to the development of database applications for "micros".  In September of 1981 I formally introduced at a farmer's conference my first database application.  It was named "XUNA D.M.I.S." (a Dairy Management Information System). 

The DBMS engine in its core, as well as its DMIS application layer, had been both written In BASIC language (the only one that was available to micros in those days).  We had very stringent memory requirements, due to the limited capacity of these early PC platforms.  And yet, the application sported: ( a ) full color,  ( b ) graphic capabilities, ( c ) efficient high-density record storage despite low-density storage medium, ( d ) scheduled and pre-defined reporting, (e) pre-defined queries, ( f ) querying tools and universal report generation capabilities, ( g ) statistics, and... a lot more.    Emphasis was put on ( h ) user-friendliness, or it would not have the appeal to the intended clientele (dairy farmers).  At the time, it was considered -by officials in that industry- to be the most powerful herd-administration tool, and decision support system (DSS), a farming operation could have. For more details about this venture please visit the page titled "Pioneer".  

Ten years later I founded another company ("Animal Science Software International", Two Rivers, WI), and in September of 1991 I introduced to farmers in Wisconsin a PC version of the DMIS, this version was named "Decade D.M.I.S.".  I developed  the database engine in C-language.  Windows was not yet what it was to become, and -furthermore- for future migration to other non-Windows mid-range platforms I chose "Vitamin C"  to build the GUI, as well as to create the DMIS application on top of the C-engine that I had to build (in a DOS platform).  (The Vitamin C library that was very popular in those days for mini-computers)

Again, for lack of venture capital (and despite great accolades from farmers and trade publications) I had to close down operations within a couple of years of having found these entrepreneurships.  I had to also deal with a divorce, and the separation from my children.  During the late 70's I had worked with radars, and data-communications equipment.  In 1992 I went back to my Telecommunications roots and finished a M.S. degree on that engineering field at the University of Colorado, Boulder.  Then, I tried to earn a Ph.D. degree in Computer Sciences from Colorado State University, Fort Collins.  But finances and child-support payments caught up with me, and -at the end of 1993- I had to abandon those studies after having earned only 1/3 of the total required credits for that program.

Since early 1994 -and until May of 1997- I worked for LODGENET ENTERTAINMENT Inc., in Sioux Falls, SD, creating their international Operations.  Hence, I got disconnected from software development.

I had been in Florida for about a year, trying to start an International Consulting and Import/Export business, when in  June of 1998 I accepted an offer by INDEPENDENT TECHNOLOGIES Corp. (ITC).  They needed a person to start their international operations, but they wanted me to work in software development first.   Sprint ( the telephone company), had insinuated to ITC they urgently needed a Data Warehouse application to consolidate all their payphone's contracts and the issuance of corresponding commissions.   I had attended the job interview with a laptop, in which I have loaded the database software I had created for farmers.  It was running under DOS (with Vitamin-C windows), yet it is a powerful engine and application.  ITC's President (Daryl Ingalsbe) realized that  I could be the one who could make the "Contracts and Commissions" Project a reality .  But, I was "rusty" I had admitted plainly to them.

DATA WAREHOUSING PROJECT

HUMAN RESOURCES.-

* Originally, I thought I was going to be a Project Leader, who would only coordinate  the development efforts.  Finalize the specifications with the Client, form the team, and direct the brain storming sessions.  But ITC was quite small, we had a total of about 100 persons on the company’s payroll, and that includes people in all four plants.  Of these, perhaps as many as 20 are hardware engineers, and only 3 were software engineers, myself being counted.  We were told, that there was no budget to hire more software people.

* One of the software engineers was pretty strong with the Visual Fox-Pro API, but did not do programming in  C++.  He had just finished migrating and enhancing a main ITC software package (called Wesroc®  was now a quite modern File-Server FoxPro application.  Unfortunately, the code at this time was still riddled with dozens of minor bugs.  Therefore, this engineer would have his hands full for a while, and could not be of much help.

* The other software engineer was pretty good in Visual C++ .  The problem was that his support to the hardware engineers was of critical importance (in our test-sets manufacturing facilities).  He could be very helpful answering quick Visual C++ questions, but we could not count with him for much more than that.   I have to admit that without his help, it would have taken me a lot longer to get up to speed with Visual C++.

* I was the third software guy.  My strength was the development of database engines in C, as well as interpreting the business rules and doing the code implementation of the business application.  But I had never used Visual C++.  When C++ was being born -in the late 80’s- I was one of those who studied it, and to prove it I have a pioneering diploma in C++ (from the University of Colorado, Fall of 1989).  But other than that, I did not have much real practice on that language, despite its closeness to C. 

THE PROJECT.-

It is called C&C (meaning Contracts and Commissions).   C&C can be looked at as the first stage of a full blown DATA WAREHOUSE system.  Hundreds of thousands of revenue/charge streams are loaded into  C&C® from several DATA MARTS (such as CDR -  Call Detail Reports, and other mainframe repositories).  The raw data is "dirty", therefore  C&C® has a built in ETL module (Extraction, Transformation and Performance) with automatic cleansing tools.  Mainly, it (a) extracts data from banks; (b) validates and tags it (a different commission rate could be applied later to each class), and removes inconsistencies; (c) then logs the purified data on a DATA STORE.  Users can now apply its built-in processing tools to calculate the commissions multi-dimensionally (that is, on a phone basis, on a contract basis, or on a payee basis).  A contract can have any number of phones.  A contract's commission may be payable to none, one, or many payees.  Furthermore, a payee can be beneficiary of commissions paid on several contracts (what in relational database terminology we refer to as a many-to-many table-relationship).  C&C® offers querying tools to analyze results. It also has many predefined queries and reports. C&C® thus becomes an intricate part of the Division operations, as well as a  DSS (Decision Support System) to the Management of this Payphone Division.  On a monthly basis, commissions' results are automatically loaded onto several logs (repositories) from where they can be analyzed with data mining tools.

C&C® -integrated with our Wesroc® product  (a Payphone Management and Administration System)- then becomes a full DATA WAREHOUSE for the Payphone Division.  Not just the Sales Department can benefit, but also Marketing, Parts Inventory, Repairs and Box-Collection Dispatching, Product Pricing (commission tiers' homogeneity across geographical areas), and Control of Accounts (Payable and Receivables).

THE REQUIREMENTS.-  

We had a brainstorming session in Omaha on the specifications, and we came up with the following resolutions to accomplish the project goals: 

1.      Could not be done with a file-server, it had to be client-server platform.

2.      Oracle would be the DBMS engine.

3.      We would not use that vendor's API (the application program interface called Oracle Enterprise). Instead we will use C++ to write the client part of this application and its GUI (Visual C++).

4.      It will not be a 2-tier DBMS application, limited to work only with an Oracle engine.  It would be an n-tier that could use middleware and ODBC to be able to connect to any other server.

5.      Since we did not have more software engineers, the “team” would be made up of just one person, the reluctant Juan Xuna.

6.      Then the boss, Daryl Ingalsbe, added the last requirement: we have to have this by "yesterday".  Meaning: ready for beta test in 3 months, and operational in 6 months.

THINGS I HAD TO DO, QUICKLY .-

1) Become a GUI "button-boy" in days.

2) Buy a lot of technical books, and read non-stop for the weeks to come.

MY MANY HATS, IN THIS “ONE-PERSON TEAM” OXYMORON.-

1)      Project Lead – Who would commit to aggressive and achievable milestones and dead lines.

2)      SME (Subject Matter Expert) – Who would need to expand quickly his knowledge of Telco’s CDR (Call Detail Reporting), to be able to discuss with the client-company the most subtle details of application’s usage.

3)      Business Analyst – Who would make sure the business rules are correctly interpreted and implemented.  And that the program has all the needed features, some of which may not even have been contemplated on the original specs.

4)      Data Modeler Designer - At ITC we do not have CASE tools (Computer Aided Software Engineering), such as Popkin or Rational Software.  Therefore the building of the ERD (Entity Relationship Diagrams) and all the METADATA had to be done the old fashion way… by brain , of course, with some help from some utilities built-in on the Microsoft SDK framework (Systems Development Kit).  Please, see at the bottom of this page what metadata is all about, and what entails.

5)      Data Loader – Although I could count on help from administrative personnel at the Client's premises, for that support to be effective it would require from me to be in close proximity with them.   Therefore, I relocated to an office at their headquarters in Overland Park, KS.  During the first stages of development though, the client had not seen the program yet, and this developer had to be data loader as well.

6)      Application Programmer – No question.  I had to come up to speed with C++, and acquire adequate dexterity and knowledge on MFC (Microsoft Foundation Classes) and Visual C++ in no time. Also I had to refresh my SQL, which was needed to write embedded code, as well as "stored procedures" that would trigger execution of processes directly on the server, making the client thinner, what improves performance.

7)      Application Tester - I would have to do the TPC Benchmarks (Transaction Processing Performance).  Design the tests, and create the test data-sets that would exercise -if possible- all branching of the design flowchart (that would be a dream on this case).  And do all the time-consuming analysis of test results.  Indeed, very tedious and labor intensive functions, which would normally require -a least- one full-time engineer to do these tasks alone.

8)      Technical Writer – Regardless of how friendly the application is, and how well documented the source code is, a trail of documents have to be created a long the way (Progress Reports).  And -finally- a User Manual would have to be written for the Client to understand how the program operates, on how it is that it meets and complies to its specifications.

9)      Trainer – When all of the above has been done, I had to be ready to give training seminars on the program, not just to users on the Client's premises, but also submit to code reviews (accolades and criticism) from peers at ITC headquarters, so you can learn from their input, and they can learn from your achievements.

This report could be extended to cover several pages. I will not do that out of respect and obligation to my Employer and his Client.  But, having covered the most significant topics –from a Résumé perspective- I will end here.  For operational details, when called to interviews I might take with me a laptop to show C&C® running (with a set of test data ).      Thanks for having read this page.

My library (83 books) on Java, C++, ODBC and so forth ...  C++Diploma (University of Colorado) dating back to 1989

P.S.  What is METADATA? :    METADATA is the data about the data!

There are 3 types of METADATA.  Some of the framework utilities used these days may help creating such documentation.  However, the developer still has to analyze the project and come up with the set of tables (scheme), the tables’ fields, and the relational links among tables.  On a project of this magnitude, simple things such as the naming of these tables and fields could be of major importance.  Unless you strictly follow a robust naming convention, pretty soon  you could be in a messy and nightmarish imbroglio. 

As with many other OLAP applications (Online Analytical Processing), and popular business applications such as ERPs (Enterprise Resource Planning – PeopleSoft, SAP, Baan), having a good naming-convention could also help tremendously the users, and your software colleagues that may have to continue maintaining what you have created.

1)      Business Metadata:  Data Source Systems, Data Synonyms, Data Access Restrictions, Valid Values, Business Rules

2)      Database Metadata:  Database names, Tables names, Column names (fields), …

3)      Application Metadata:  Reports, Queries explanations.

 

    

Juan Xuna

Miami Beach,  FL 33154 

Hit Counter   Due to problems with FrontPage  Extensions,

counter had to be reset on 08/22/2012

               

 Phn: (305) 600-2340     

   Fax: (561) 210-1370    

Email: Xuna@MSN.com