Minimize Project Risk and Costs: A New Approach to Project Management

 

Traditional project management (PM) results have been poor. The practices of direction and control have been identified by deductive logic as problematic. Deductive logic identifies that, if a manager directs and controls, their risk goes up and performance will go down. A new approach to PM is the replacement of technical expertise with the identification and utilization of expertise. New components of this approach are the minimization of the need to think and make decisions, the use of the language of metrics, a new definition of risk, and the use of preplanning that includes the utilization of expertise and focusing on the mitigation of risk that the expert does not control. This approach has been tested by the International Council for Research and Innovation in Building and Construction Working Commission (CIB W117, formerly known as the Performance Based Studies Research Group at Arizona State University) for the past 25 years increasing customer satisfaction to 98% and minimizing cost by 5 to 30%. These practices are a part of the “PM of the Future.”

Keywords: Best Value, Project Management, Logic, Construction, Delivery of Services.

Introduction

According to researcher publications, project management (PM) is a fundamental key component of delivering services successfully in terms of on time, on budget, with high customer satisfaction (Sears, et al., 2008; Anantatmula, 2010; PMI, 2013; Dinsmore, 2014; Rivera, 2017). The Project Management Institute (PMI) conducted a study in 2010 that identified the importance of PM across multiple industries (PMI, 2010). Ninety percent of global senior executives ranked PM methods as important to their ability to deliver successful projects and remain competitive (PMI, 2010).

PM positions have a major impact on delivering services efficiently and effectively. By observation, with the growing demand of delivering services in multiple industries, PMs will be expected to know more and have more experience to survive, making an already difficult job, more difficult (Rivera and Kashiwagi, 2016). With the continued poor performance seen in these industries, it is difficult to see how PMs who manage, direct and control the delivery of services will ensure they receive high performing and successful projects.

This research paper proposes that the current PM models are not helping in delivering high performing projects and PMs will have to use a new model to improve the performance of their projects.

Project Managers’ Delivering Poor Performing Projects

The performance of construction services has been relatively higher than other industries such as Information Technology services (Rivera, 2017). However, studies have identified that even construction services do not have high performance. The International Council for Research and Innovation in Building and Construction Working Commission 117 (CIB W117), is a world-renowned research effort (formerly known as the Performance Based Studies Research Group at Arizona State University), headed up by retired professor Dean T. Kashiwagi, and has been conducting PM research and development for the past 25 years. In 2013, CIB W117 sanctioned a worldwide literature research study to identify its effectiveness in the construction delivery of services (Rivera, 2014, Rivera, et.al., 2016a). Relevant publications were found by viewing abstracts from one of four research databases (ASCE Library, Science Direct, Taylor and Francis Online, Emerald Insights). In total, 3,200 publications were sifted through, 260 were found to be related to the construction performance topic and were reviewed in more detail. After further review, 95 had documentation of performance in terms of rework, cost and schedule overrun, and customer satisfaction.

Table 1 shows the breakout of performance from 38 countries identified and grouped into six major regions worldwide. After analyzing the 95 construction publications, the results were documented in Table 1.

The study results identified that construction service performance in all regions and countries have delays and a significant change order rate. The struggles with construction performance is a major issue to both developing and developed countries, as construction services are required in all industries. Dr. William Badger (Professor Emeritus at Arizona State University and a Construction Industry Institute expert), identified that more infrastructure will be built in the next 30 years than the last 2,000 (CII, 2015).

In 2016, another study was sanctioned by CIB W117, to compare service performance from all industries that employ PMs. Relevant publications were found by viewing abstracts from major research databases (NASA, JSTOR, Emerald Insight, ProQuest, ASU Library, Science Direct). In total, 208 publications were sifted through, 105 were found to be related to the project management topic and were reviewed in more detail. After further review, 36 had documentation of performance in terms of cost and schedule overrun, customer satisfaction, and quality. The results (see Table 2) identified that the delivery of services in all industries researched were poor (Rivera, 2017; PBSRG, 2017). The current way PMs are delivering services is not able to deliver high performance results.

Research has identified that the performance issues that PMs are facing could be caused by the following factors (Ahern, et al., 2014; Elonen, et al., 2003; PBSRG, 2017):

1. Services are too complex.
2. Increased number of supply chain participants and silos.
3. Unclear roles of participants.
4. Confusion over details, which increases decision-making.
5. Misunderstood client expectations.
6. Increased misunderstandings.
7. Required to know every detail on a project.
8. Poor preplanning.
9. Project environment is non-transparent due to a lack of performance measurements.
10. Reactive environment due to client management, direction, and control.

The CIB W117 has worked with 123 clients over the past 25 years to identify the cause of poor performance in the delivery of services. The research it has performed proposes the problem might be that the current PM models create complexity on a project, causing risks to impact the performance of the service. The PM models are generated from creating and sustaining silos in service delivery. In most organizations, there are three different silos for delivering services:

1. Operational – the group that needs the service.
2. Procurement – the group that buys the service.
3. Project management – the group that manages the delivered services.

The first group attempts to identify what they need, the second group buys the service and the third group manages it. It has been observed that none of the three groups understands fully what is being done in the other silos. Logic identifies that silos create complexity, which generates confusion and misunderstandings. All three areas lack sufficient expertise and knowledge to deliver the service themselves. One attempts to identify what they want, the second group attempts to identify the best value service through competition, and the third group attempts to manage the expert who is delivering the service.

The silos are created and sustained because all three organizations have a goal to: 1) be more professional, and 2) to show they have value. This is done through the following actions:

1. Joining professional organizations that are in their area.
2. Requiring technical degrees in their area of expertise.
3. Attempt to create technical standards that only they understand, and then create certification in those areas to show that they have value.
4. Require experience and technical education to gain the certification.

As organizations age, the following is true about their characteristics:

1. Become more bureaucratic.
2. Are more difficult to change.
3. Their silos are strengthened.
4. Forces their “certified individuals” to continually know more technical information about everything in this fast-changing environment.
5. Certified individuals take on the role of the expert and manage, direct and control (MDC) the service provider.
6. MDC creates more complexity, which leads to lower performance.

Proponents of sustaining professional silos that create complexity in service delivery include: the National Institute of Government Purchasers (NIGP), Institute of Supply Change Management (ISM), Project Management Institute (PMI), International Project Management Association (IPMA) and the International Facilities Management Association (IFMA). With the tremendous amount of documentation on how to be more professional and more valuable, the delivery of projects in all these areas have continued to suffer poor performance in their service delivery.

Proposal

A new PM model is being proposed, which was devised from the Best Value Approach (BVA) methodology. The BVA is the same methodology that developed the Performance Information Procurement System (PIPS) and the Performance Information Risk Management System (PIRMS). These systems have been continually developed over the last 25 years to help organizations overcome the complexity generated by their internal silos (see Figure 1). The two systems have had the following performance in delivering services (Rivera, 2014; PBSRG, 2017):

1. Established in 1992 and has documented performance on over 2000 projects and services delivered (construction and non-construction).
2. $6.6B of projects and services delivered with a 98% customer satisfaction and 9.0/10 client rating of process.
3. Services delivered: construction, facility maintenance, IT, professional (design), redesign of systems and organizations and supply chain applications.
4. $17.6M in research funding generated, due to the effectiveness of decreasing buyer cost of services on average by 31% (57% of the time, the highest performing expert was selected and was the lowest cost).
5. Contractors/experts could offer the client/owner 38% more value, and decreased client efforts by up to 79%.
6. 90% of all project cost and schedule deviation is caused by the owner’s non-expert stakeholders.
7. Change order rates were reduced to as low as -0.6%.
8. CIB W117 has worked with over 123 unique clients (both government and private sector) and received 12 National/International Awards.
9. A PM who identifies and utilizes expertise can increase production by 10 times.
10. 5 to 30 percent cost savings are achieved on the projects.
11. The BVA is the most licensed technology to come out of Arizona State University licenses (54).
12. It is internationally recognized through repeated testing (Canada, Netherlands, Sweden, Norway, Finland, Botswana, Malaysia, Australia, Democratic Republic of Congo, France). Education efforts are in Poland, Saudi Arabia, India, Vietnam and China.
13. Some of the largest projects documented were: $100M City of Peoria Wastewater Treatment DB project (2007); $53M Olympic Village/University of Utah Housing Project (2001); $1B Infrastructure project in Netherlands (2009).
14. Some of the highest performing projects documented include: ASU tested BVA in their business services and procurement department, resulting in $100M of revenue. Changed the entire procurement service industry in the Netherlands through the success of a $1B infrastructure test that cut procurement cost by 50% and help the project finish 25% faster. As a result, the Rijkswaterstaat won the most prestigious procurement award in the Netherlands, the 2012 Dutch Sourcing Award, and now NEVI (Dutch Professional Procurement Group) is licensing BVA technology and certifying in the Netherlands (Rijt, J., Santema, S. 2012).

The PIPS/PIRMS have been audited multiple times in the last 25 years. Two of the audits identified the impact and effectiveness of the PIPS/PIRMS systems in detail:

1. The State of Hawaii Audit (Kashiwagi et al. 2002; State of Hawaii Report 2002 (DISD)).
2. The Dutch Study on the Impact of PIPS (Duren & Doree, 2008).

These studies confirmed all performance claims of the systems were accurate. Duren and Doree’s study found the following results for projects performed in the United States (Duren & Doree, 2008):

1. 5% of clients who worked with PIPS/PIRMS identified that their projects were delivered on time.
2. 7% of clients who worked with PIPS/PIRMS identified that their projects were delivered within budget.
3. 91% of the clients stated that there were no charges for extra work.
4. 9% of the clients awarded the supplier’s performance with greater than an 8 rating (on a scale from 1-10, 10 being the highest performance rating).
5. 94% of clients would hire the same supplier again.

The other two groups that conducted audits were The Corps of Engineers (COE) PARC, 2008; 2008; and the Western States Contracting Alliance (WSCA) Agreement, 2011. No other research group has had their research test results audited (Kashiwagi, 2016; PBSRG, 2017).

Movement toward Automation, Robotics and Information Systems

There is a movement of automation going on in our environment. Removing the human worker along with their thinking, decision making and illnesses, has increased the quality and consistency of technologies being developed. In the future, the following three areas will have value:

1. Anything that can be automated will be automated.
2. The identification and utilization of expertise.
3. Areas of expertise that cannot be automated.

The area which offers tremendous potential to project managers is the identification and utilization of expertise. In the last 25 years of research with the BVA, the ability for project managers with no technical expertise to identify and utilize expertise of expert vendors to deliver outstanding performance shows a great area of impact for project managers. The researchers propose that project managers in areas of poor performance, may increase the performance by using the BVA as a project management model.

Proposed Best Value Approach Project Management Model (BVA PM)

A project manager will be able to utilize the new BVA project management model to improve their ability to deliver services. This will have some immediate impacts on the delivery of services (see Figure 1):

1. First, it eliminates an organization’s main service delivery silos (operations, procurement, project management), by bringing in and allowing the PM to lead the project from inception to completion. The PM will identify the expert, then utilize their expertise to identify the requirement and deliver the services.
2. Second, during the selection of a project, it creates a decision-less structure (BVA structure) for the PM to identify and utilize expertise.
3. Third, during a clarification phase, it shifts all accountability of project success from the PM to the expert vendor. This is done by requiring the expert delivering the service, to utilize their expertise to create transparency by seeing into the future. The expert will create a plan that clarifies the operation group’s expectations, how they will accomplish the plan, all stakeholder roles, responsibilities and resource requirements, and how they will measure and report success. This will be approved by the PM prior to a contract award.
4. Fourth, during the execution phase, it minimizes the workload of the PM by replacing the PM’s normal quality control responsibility with a non-technical quality assurance responsibility. The PM will collect a weekly report from the expert that identifies the current progress, performance and risk occurring on the project.

This new PM model will be known as the Best Value Approach (BVA) Project Management Model (BVA PM).

What makes the implementation of the BVA PM model difficult, is that it requires the PM to lead both the procurement and PM functions (see Figure 1). It affects two different professional silos, but when implemented is more impactful (Rivera, et al., 2016b; Rivera and Kashiwagi, 2016). In the proposed new BVA PM model the PM has the following responsibilities:

1. Minimize silos.
2. Does not manage, direct or control another entity.
3. Main responsibility is to identify and utilize expertise.
4. Has the expert (hired expert) create transparency, to help all non-experts (all stakeholders interfacing with project that is not the expert) to see the future of the project.
5. Identifies the expert as one who can see from the beginning of the project to the end.
6. Identifies that an expert has no risk.
7. Identifies that risk is caused by those who cannot see into the future (non-intentional and predictable), and their risk can only be mitigated by an expert.
8. Requires that all efforts must be measured. Measurements must be simple, understandable, and non-technical. Measurements enable the PM to know the level of performance of the expert.

The BVA PM identifies that there are PMs who will be experts (service providers) and others who will not be experts (PMs who manage the service providers). The PMs who are not technical experts must be experts in identifying and utilizing expertise.

It proposes the following new definitions for the BVA PM:

1. Expert: Someone who can see from beginning to end of a project. Someone who creates minimal risk (for operational purposes an expert has no risk).
2. Risk: Caused only by non-expert stakeholders. Risk is not intentional. Risk is predictable. Risk can only be mitigated by an expert.

Observation of the environment of the delivery of services, identifies the following:

1. Any service that can be automated, will be automated.
2. The only services that cannot be automated, are where experts are required.
3. Experts are very few.
4. Experts can identify non-experts.
5. The communication of experts to non-experts is very simple and concise.
6. Most stakeholders are non-experts.
7. Experts always deliver high performance services (high quality) for the lowest possible price.

By observation, most PMs are traditional and do not accept these new concepts, but any visionary stakeholder or PM who is reading this publication can implement these concepts as test cases. Test cases will not cause the traditional PMs and associations to be afraid of change. To be effective with implementing innovative solutions, especially organizations which are large, must always run tests that show the potential for lower costs, lowering of risk and stress, and higher performance. Most stakeholders, including PMs, are non-observant (opposite of experts), are bureaucratic and are trying to survive. The BVA PM can be termed as “disruptive” and must be treated with care.

The BVA PM will create efficiency and effectiveness. It will optimize the contribution of experts. Experts will always preplan and mitigate risk before they deliver a service.

The following sections will identify the difference between the BVA PM and traditional PM models.

Traditional PM Model vs. BVA PM Model

To clearly show the difference between the traditional project management models and the BVA PM model, another literature research was performed. This research was performed as part of a doctoral student’s dissertation, from 2015-2017 (Rivera, 2017). The literature research compared the BVA PM Model practices and characteristics to the top Traditional project management models.

The research looked into many different research databases. The research first identified all PM models that had been documented in a publication. Then the research identified the PM models that had objective performance information. Objective performance information being defined as non-survey data collected on projects (e.g. cost savings, on budget/schedule, and any dominant information). Then the research compared those traditional models with objective information with the BVA PM model.

Out of the 10,503 publications searched, 800 were reviewed in more detail. Out of the 800, 572 publications were used for the research. The researcher identified 19 traditional PM models in total to investigate. Out of the 19 traditional PM models, only 10 models had objective information. Once identified, a more in-depth analysis was performed on them to determine the practices that the models used to improve the performance of the delivery of services.

The literature research supported the CIB W117 research group’s 25 years of industry observations and documentation that the traditional project management model includes the following major three characteristics and the practices that go with each:

1. Management, Direction and Control – PM telling the expert what to do, which consists of the following PM practices:
   1. Choosing which expert should conduct a project in selection.
   2. Determining the scope, cost, and schedule of each project based upon their technical experience and understanding prior to or after the contract has been awarded.
   3. Directing the expert during the execution of the project, on how to carry out the project to completion.
   4. Directing the expert on what to do if something is perceived as not being done correctly, or unforeseen occurs on the project.
2. Technical Communication – any form of communication (verbal, written or tool) that was detailed and complex, which requires technical training/education or knowledge in a certain area to understand. This characteristic consists of the following PM practices:
   1. Expects to know all the technical details of project.
   2. Expects to discuss the technical details and detailed plan with the expert suplier.
   3. Required to make technical decisions on the project.

3. Quality Control – any inspection or verification of the quality of the experts’ work, which consists of the following PM practices:
   1. Inspection of the expert’s work.
   2. Explaining to the expert why a part of their work is not acceptable, did not meet the technical standard and ensure that they fix the work in a timely manner.

The BVA PM replaces MDC, technical communication and quality control with the following practices:

1. Leadership – PMs identifying and utilizing the expertise of their experts to deliver services.
   1. Use a rating system in selection that prioritizes the best value, by requiring the experts to show how their expertise, in terms of metrics from similar projects, compares with the current project needs and their ability to explain it simply.
   2. Require the best value expert (highest prioritized expert in selection) to determine the scope, cost, and schedule for the project.
   3. Require the expert to identify how they will carry out the project and what to do if something unforeseen occurs on the project.

2. Non-Technical Communication – any form of communication that is simple and easy (written, verbal, process or tool), which does not take previous education and training or knowledge in an area to understand.
   1. Require the expert to know all the technical details of the project, and have a detailed plan from beginning to end.
   2. Require the expert submit a weekly project report that identifies at a high level the current progress, performance and risks that occurred and how they will overcome them.
   3. Require the expert to make all technical decisions on the project, and provide a simple explanation if there is any cause for concern.

3. Quality Assurance – ensure that the expert has a plan before they begin a project, and they can explain the progress and changes to the plan throughout the execution of it.
   1. Require the expert to have a quality control plan that is verifiable through metrics and can be reported on as frequently as the PM needs it.
   2. Require the expert to identify how they will fix any work that is perceived not acceptable or does not meet a technical standard, and the time and cost deviation.

The major shift from the traditional PM model to the BVA PM model is the replacement of the PMs’ expertise with the expertise to identify and utilize expertise. The expert will create the project plan from beginning to end, and identify how they know it will be successful in terms of metrics acceptable to all stakeholders prior to award. It is shifting from being the expert to utilizing expertise. Table 3 identifies a high-level summary of all the major changes in practices (Rivera, 2017).

The success of the traditional model is based off a PM’s expertise and experience, whereas the BVA PM shifts all accountability of success to the expert.

State Agency Case Study

The BVA PM was tested at an environmental State Agency. The State Agency is one of the United States’ top 18 largest states, serving a population in the millions.

Over the last decade, the State Agency has had difficulties with the performance of their environmental professional services. They identified the following problems:

1. Unable to identify performance and value of service providers (environmental experts).
2. Vendors (experts) were not meeting the quality expectations of the State Agency.
3. Management requirement of the experts was too high.
4. Inability to spend all available resources.

In FY2015, the researchers were authorized by the State Agency to conduct its first test of the BVA PM in their largest department, Water Quality Department (WQD) (Rivera, 2017). This department is responsible for identifying, assessing, and cleaning up soil, groundwater, and surface water sites contaminated with hazardous substances throughout the state with support from state funds. In other words, the WQD identifies polluted underground water and cleans it up. The program also oversees privately funded cleanup efforts.

The WQD was made up of the following components:

1. 2 upper management personnel (Section and Unit manager).
2. 5 project managers (each has 15-30+ years of experience in environmental services, and holds at least a bachelors in geology, hydrology or environmental engineering. No other information was collected on the project managers).
3. Use an indefinite delivery indefinite quantity contract (IDIQ) (multi-year contract that uses multiple service providers to deliver services).
4. 26 water contaminated sites throughout the state.
5. Have another 28 sites that need to be verified for water contamination.
6. 10 service providers on the IDIQ contract.
7. $7M budget for FY2015.

 FY2015 Case Study Test Results

 The WQD Unit Manager identified the BVA PM as a success. Although the State Agency attempted to implement the BVA PM on their projects, traditional practices of management, direction, and control was still a factor. Despite the use of traditional practices, Table 4 identifies the following observations.

1. WQD project managers increased work capacity by 43% as a whole (calculation is ((FY15 Total cost / # of PMs) – (FY14 Total cost / # of PMs)) / (FY14 Total cost / # of PMs)). The individual hours spent by each project manager from the previous year and test year was not collected.
2. WQD received more work (98% more scope of work completed by experts) in 33% less time (more work was measured by identifying the percent change of scope of work completion from the previous year to the new year (calculation equals: similar amount of work ($5.5M vs $5.6M) and there was a 98% increase in amount of work done (50% à 99%)). 33% less time was the amount of time the experts received less to complete their work then the year before. Experts were able to start work in August the year prior, and the year of the test, they started in November).
3. WQD project management satisfaction of the quality of work produced increased by 22% (measured by taking the percent increase of satisfaction from the previous year to the test year (calculation equals: (FY15 satisfaction rating – FY14 satisfaction rating) / FY14 satisfaction rating)).
4. Similar number of projects were conducted at similar amounts of cost.

The BVA PM did the following for the State Agency WQD:

1. Identified and utilized experts to deliver services.
2. Required experts to take full control over the project and become accountable for it.
3. Required the experts to create a plan that included performance metrics to identify how they knew their projects would be considered a success.
4. Implemented a new project report that measured the schedule and cost deviations of a project on a weekly basis, to include who was the responsible party.

The BVA PM has identified the following observations:

1. Successful implementation shows that the BVA PM may be a resourceful solution for a PMs to use to manage and receive high performing services.
2. Using the leadership practice of identifying and utilizing expertise to lower cost and increase performance appears to be an impactful idea.
3. An expert may be able to identify a project’s scope and cost more accurately then a PM.
4. Measurement brings transparency and minimizes decision-making.
5. The BVA PM has been identified by the State Agency as the only option proven to transform its agency’s environment from a management, direction, and control to an alignment, win-win, transparency and leadership based environment.

The State Agency Water Quality Division (WQD) has continued to implement the BVA PM. The BVA PM is an advanced and theoretically sound new PM model that can transform the traditional PM model from a silo-MDC based environment to a leadership based environment.

Conclusion

A new PM model has been researched and developed over the past 25 years by the CIB W117 research group. The new PM model is called the Best Value Approach Project Management model (BVA PM). The BVA PM helps organizations eliminate the silos of service delivery which causes complexity and low performance, by bringing in the PM to lead the project from inception to completion. The PM replaces the traditional practices (MDC, technical communication, quality control) of delivering services with the implementation of the new BVA PM practices (leadership, non-technical communication and quality assurance).

The BVA PM’s major contribution to minimizing cost and risk and increasing performance is through the replacement of the PMs’ expertise with the identification and utilization of an expert service provider’s expertise. It is a new paradigm shift for PMs to recognize they are not the experts. The experts are those who are performing the service, can see into the future and make it simple. The new BVA PM also lowers cost, by minimizing the actions of non-expert stakeholders. The new BVA PM has been tested in a large government agency to show high performance and client satisfaction. The researchers propose that the BVA PM is the PM model of the future to support PMs with the delivery of services. For more information, see the referenced documents.

References

• Ahern, T., B. Leavy, and PJ Byrne. (2014). “Complex Project Management as Complex Problem Solving: A Distributed Knowledge Management Perspective.” International Journal of Project Management 32.8: 1371-81.
• Anantatmula, V. S. (2010). Project manager leadership role in improving project performance. Engineering Management Journal, 22(1), 13-22. Retrieved from http://login.ezproxy1.lib.asu.edu/login?url=http://search.proquest.com/docview/734620101?accountid=4485.
• CII. (2015). CII 25 – Building on 25 Years. Construction Industry Institute. Web. (2 October 2015). Retrieved from https://www.construction-institute.org/scriptcontent/more/cii_25_more.cfm
• CII. (2015). Performance Assessment 2015 Edition. Construction Industry Institute. Web. (2015). Retrieved from http://www. Construction-institute.org/performance.
• Dinsmore, P., and Cabanis-Brewin, J. (2014). AMA Handbook of Project Management (4th Edition). Saranac Lake, NY, USA: AMACOM Books. ProQuest ebrary.
• Duren, J. and Doree, A. (2008) An evaluation of Performance Information Procurement System (PIPS), 3rd international public procurement conference proceedings 28(30) pp 923-946.
• Duren, J. and Doree, A. (2008) An evaluation of Performance Information Procurement System (PIPS), 3rd international public procurement conference proceedings 28(30) pp 923-946.
• Elonen, Suvi, and Karlos A. Artto. (2003). “Problems in Managing Internal Development Projects in Multi-Project Environments.” International Journal of Project Management 21.6: 395-402.
• Kashiwagi, D, Kashiwagi, J, Rijt, J., Santema, S. (2012). The Best Value Approach in the Netherlands: A Reflection on Past, Present, and Future, PBSRG 2012 Journal for the Advancement of Performance Information and Value VOL. 4 NO. 2.
• Kashiwagi, D. (2016). Best Value Approach. Performance Based Studies Research Group. Tempe, AZ. Publisher: KSM Inc.
• Kashiwagi, D. (2017). How to Know Everything Without Knowing Anything. Kashiwagi Solution Model Inc. Mesa, Az. Publisher: KSM Inc.
• Kashiwagi, D., & Kashiwagi, I. (2014). The Best Value ICT Industry. Journal for the Advancement of Performance Information & Value, 6(1).
• Kashiwagi, D.T., Savicky, J. and Kashiwagi, A. (2002) “Analysis of the Performance of ‘Best Value’ Procurement in the State of Hawaii” ASC Proceedings of the 38th Annual Conference Virginia Polytechnic Institute and State University – Blacksburg, Virginia, pp. 373-380 (April 11, 2002).
• PBSRG (2017). Performance Based Studies Research Group. Retrieved August 2017 from PBSRG Web site: http://pbsrg.com/overview/documented-performance/
• PBSRG.com. (2017). Academic and Research Papers. Performance Based Studies Research Group, Arizona State University. Retrieved from http://pbsrg.com/publications/papers-intro/.
• PMI (2010). The Value of Project Management White Paper. Project Management Institute. Web 4 April 2017. PDF.
• Project Management Institute (PMI). (2013). A Guide to the Project Management Body of Knowledge (PMBOK Guide). 5th ed. Newtown Square, Pa: Project Management Institute, Inc., 2013.
• Rivera, A. (2014). Master’s Thesis, M.S. “Impact of a Non-Traditional Research Approach Case Study on the Performance Based Studies Research Group (PBSRG).” Arizona State University.
• Rivera, A. (2017). Dissertation, Ph.D. “Shifting from Management to Leadership: A Procurement Model Adaptation to Project Management.” Arizona State University. ProQuest Dissertations Publishing, 2017. 10681244.
• Rivera, A., & Kashiwagi, J. (2016). Identifying the State of the Project Management Profession. Procedia Engineering, 145, 1386-1393.
• Rivera, A., Kashiwagi, J., & Kashiwagi, D. (2016b). Improving the Management of Environmental Engineering Projects through the Best Value Project Management Model for a State Agency. Journal for the Advancement of Performance Information & Value, 8(1).
• Rivera, A., Le, N., Kashiwagi, J., & Kashiwagi, D. (2016a). Identifying the Global Performance of the Construction Industry. Journal for the Advancement of Performance Information & Value, 8(2).
• Sears, S. K., Clough, R. H., & Sears, G. A. (2008). Construction project management: A practical guide to field construction management. Hoboken, N.J: John Wiley & Sons.
• State of Hawaii PIPS Advisory Committee (2002), Report for Senate Concurrent Resolution No. 39 Requesting a Review of the Performance Information Procurement System (PIPS), Honolulu, HI: U.S. Government, Available from: http://Hawaii.gov/dags/rpts/pips.pdf>.