1
|
|
2
|
- Introduction to the IRS
- Overview of Business Systems Modernization (BSM) Program
- MITRE’s IRS FFRDC Role
- Background on Process Improvement (PI)
- Application of the Software Acquisition Capability Maturity Model (SA-CMM®)
- Measurement Aspects of SA-CMM® and PI
- Experience So Far
- Next Steps
|
3
|
|
4
|
|
5
|
- In December 1998, IRS awarded the modernization contract to the PRIME Alliance for up to 15
years
- IRS BSMO is the Acquisition Organization
- Solution Provider—CSC and PRIME Partners
- Program Size—$5–$8B
- BSMO current staffing—approximately 165
- Program encompasses large scale IRS systems including tax administration
systems and agency internal systems
|
6
|
|
7
|
- BSMO realizes that its performance cannot improve without process improvement
- Meets Congressional mandate to set up a “world class” acquisition organization by following
Software Engineering Institute’s
(SEI) Capability Maturity Model (CMM) methodologies
- General Accounting Office (GAO) builds on and fosters this expectation and will audit using the
SEI SA-CMM®
- Makes BSMO more “accountable, predictable and timely,” as required by
John Reece, IRS CIO
|
8
|
|
9
|
|
10
|
|
11
|
|
12
|
- GAO conducted an audit in 1998 recommending improvements in IRS systems
acquisition processes
- IRS / BSMO developed SA-CMM®-compliant processes in 2000
- January 2001 informal assessment conducted on BSMO with Software
Engineering Institute (SEI) indicated weaknesses
- Instability in Organization
made implementation problematic
- Did not completely reflect
intricacies of BSM environment
- Organizational stability occurred in early 2001
- Process Improvement (PI) Management (BSM executives) Steering Group
formed in August 2001
- Solutions Acquisition Process Group (SAPG) chartered to implement PI
- Initial Goal
- Software Capability Evaluation
(SCE) for SA-CMM® Level 2
maturity in December
2002 (will be led by SEI)
|
13
|
- The Must Haves
- 1. A compelling reason for change
- 2. Leadership of the change effort by the top executive in the
organization—responsibility cannot be delegated
- 3. Informed commitment of the top management team
- 4. Designation of a primary PI Leader and an adequate mandate for
change
- 5. Sound performance measures that drive change
|
14
|
|
15
|
- Application of the SA-CMM® helps establish the infrastructure
to better manage acquisitions (software and systems)
- Forces an “introspective” of how the organization does business and how
it wants to do business—not on how the supplier (contractor) does
business
- Key concepts of the SA-CMM®
- It targets acquisition and management
- Someone is in charge from the acquisition organization for the
management of each acquisition: a team is established
- Each acquisition is planned using specific processes and procedures
- The plan is executed and tracked
- Work is verified by senior management
- Policies, processes, descriptions, and procedures are consistent with
the SA-CMM® and the way the organization wants to do
business
|
16
|
|
17
|
|
18
|
|
19
|
|
20
|
|
21
|
- Obtained BSMO executive management sponsorship
- Identified BSMO pilot PI Management Steering Group (MSG) and Solutions
Acquisition Process Group (SAPG)
- Developed BSMO PI Strategic Plan and Schedule
- Developed Process Architecture (mapping, procedures*)
- Developed BSMO PI Communications and Training Plans
- Provided PI training to SAPG team members, projects, executives
- Developed and rolled-out 7 major processes and associated procedures*; Process Measurement is
integrated
- Conducted informal CMM Based Application-Internal Process Improvement (CBA-IPI) with SEI-led
team in June 2002 to determine status, strengths, and PI opportunities
(MITRE is part of team)
- *e.g., some procedures for Task Order Monitoring are status meetings,
deliverable reviews, cost and schedule evaluation, issue tracking
|
22
|
- Developed a Measurement Plan
- Provided PSM Executive Overview for SAPG
- Ensured each Process Description had a section on Process Measurement; also emphasized in
Acquisition Project Planning process / procedure
- Developed some typical examples: plan vs. actual progress
- Offered Measurement workshops to projects using PSM methodology; terms tailored for IRS /
BSM
- Conducted one (to date) and drafted measurement implementation plan with project consideration
- Offered consulting / guidance as needed*
- Using June 2002 CBA-IPI findings to focus on measurement activities to
improve by December 2002 SCE
- *SEI has provided consulting support to us
|
23
|
|
24
|
- Measurement Overview (why, what, how)
- Conduct Workshop exercise (PSM Methodology)
- Identify concerns (e.g., objectives, constraints, issues, risks)
- Group into similar concerns; prioritize importance
- Clarify into concrete, measurable terms (ask questions)
- If time permits, outline some possible visual graphics
- Identify any current measurements (may be applicable)
- Capture implementation issues
- Resources, responsibilities,
tools, validity, ad hoc demands
- Next Steps after initial workshop
- Draft measurement plan and review
- Assist and follow up
|
25
|
- By Organization Level
- Enterprise
- Program
- Project
- Process
- Product
|
26
|
- Executives want to know impact / return on investment
- Long term: requires stability and capability to roll-up across organizations
- Links with BSMO Enterprise Performance Management
- Most Capability Maturity Models will indicate need for certain projects
/ product measures (e.g., cost, schedule, requirements, defects, TPMs,
etc.) for particular process areas
- Common feature for institutionalization requires need for measures of
each process area (e.g., resources, cycle-time, output quality)
|
27
|
- The SA-CMM® says so: it’s a best practice!
- You can’t improve what you don’t measure
- Need objective evidence to understand
- How long something takes
- How much effort is necessary to do it right
- It’s difficult to know when you’ll get there if you don’t know where you
are or how you got there
- You need a baseline for planning and estimation
|
28
|
- Credible, objective evidence to make your life easier
- Demonstrate successes
- Identify critical issues / risks early
- Guide difficult decisions
- If it’s not used, don’t bother collecting it
- Collect it, store it, manage it
- Analyze it, report it, present it
- Review it and take corrective actions as necessary
|
29
|
- You need to status your process activities and products to
become effective*
- Typical examples
- Process compliance
- Schedule and cycle-time
- Effort and resources expended on process activities
- Quality defects in work products and deliverables; rework effort, time
- * not only for statusing, but
for developing a database for understanding and planning / estimation
|
30
|
- Keep it simple and practical
- Measure what you really need
- Reuse existing measures if possible
- Integrate with processes—not separately
- Let all know what is being collected and why
- Share results
- Don’t use to punish, but to inform and learn
- Measures are an aid—not a replacement for all one needs to
manage efficiently and effectively
|
31
|
- Some Draft / Preliminary Results
- Not enough time has past to implement all new processes / procedures
across full acquisition life cycle
- Some pilot projects are not yet in a phase to trigger some processes /
activities
- Some pilot projects cannot fully implement process measurements until
new processes / procedures are fully underway [e.g., stable]
- Opportunity of doing additional
measurements beyond the very simple / basic [e.g., plan vs actual
key activities] is a challenge
- A few indicate contractor
measures (e.g., EV) would relate to acquisition project’s process
measures
|
32
|
- Projects attempting Level 2 of a CMM must first be first fully aware of
new processes / procedures before process measurements can be most useful
- Start simple and small...but useful
- Limited resources dictate that only the most useful and efficient
measures be implemented (keep asking why for each desired measure)
- Collecting effort data in an organization is challenging if not part of
an infrastructure with a Work Breakdown Structure at an appropriate
level
- Some projects are schedule driven and resource constrained, and
additionally react to “ad-hoc” work requests
- This behavior illustrates need for effort, schedule, and quality
measurements to back up need for more resources and reasonable
schedules
- BSMO has traditionally depended mostly on PRIME for detailed planning
and estimation
- This recognized shortcoming is now being worked (with MITRE support)
- Again, illustrates another reason for the need for a good measurement
program and database to improve planning / estimation capability
|
33
|
- Update processes and procedures, including measurements, based on CBA-IPI findings
- Implement more training, including measurement workshops, to
institutionalize and gain further experience in repeatability
- Develop more specific measurement examples as appropriate
- Begin developing an organizational measurement database for improving
estimation and impact analysis, risk analysis, etc.
- After Software Capability Evaluation in December
- Further fine tuning, as necessary
- Begin to roll out processes / procedures to all BSMO projects
- Begin preparing for SA-CMM® Level 3 maturity goals
- Consider Systems Engineering processes (e.g., CMMI, SECM)
- Integrate Estimation and Enterprise / Program Performance Management measurement aspects into an
overall approach
|
34
|
- Lloyd Anderson, IRS Business Systems Modernization Office (BSMO) - IRS
Sponsor, PI Lead
- Don Gantzer, The MITRE Corporation - supports IRS PI program
- Matt Fisher, Software Engineering Institute - consults to IRS
- Dave Zubrow, Software Engineering Institute - consults to IRS
- Dennis Goldenson, Software Engineering Institute - consults to IRS
dg@sei.cmu.edu
- Arnold Smith, The MITRE Corporation - leads support to IRS PI Program
|
35
|
|
36
|
|
37
|
|
38
|
- As a public interest company, in partnership with the government, MITRE
addresses issues of critical national importance, combining systems
engineering and information technology to develop innovative, viable
solutions that make a difference.
|
39
|
|
40
|
|