{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T07:35:10Z","timestamp":1768894510199,"version":"3.49.0"},"reference-count":72,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2020,10,21]],"date-time":"2020-10-21T00:00:00Z","timestamp":1603238400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Software Evolu Process"],"published-print":{"date-parts":[[2021,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>DevOps is a new software engineering paradigm adopted by various software organizations to develop the quality software within time and budget. The implementation of DevOps practices is critical, and there are no guidelines to assess and improve the DevOps activities in software organizations. Hence, there is a need to develop a readiness model for DevOps (RMDevOps) with an aim to assist the practitioners for implementation of DevOps practices in software firms. To achieve the study objective, we conducted a systematic literature review (SLR) study to identify the critical challenges and associated best practices of DevOps. A total of 18 challenges and 73 best practices were identified from the 69 primary studies. The identified challenges and best practices were further evaluated by conducting a survey with industry practitioners. The RMDevOps was developed based on other well\u2010established models in software engineering domain, for example, software process improvement readiness model (SPIRM) and software outsourcing vendor readiness model (SOVRM). Finally, case studies were conducted with three different organizations with an aim to validate the developed model. The results show that the RMDevOps is effective to assess and improve the DevOps practices in software organizations.<\/jats:p>","DOI":"10.1002\/smr.2323","type":"journal-article","created":{"date-parts":[[2020,10,21]],"date-time":"2020-10-21T07:31:01Z","timestamp":1603265461000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Readiness model for DevOps implementation in software organizations"],"prefix":"10.1002","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9807-6235","authenticated-orcid":false,"given":"Saima","family":"Rafi","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology Chongqing University of Posts and Telecommunication  Chongqing China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wu","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Cyber security and Information Law Chongqing University of Posts and Telecommunication  Chongqing China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6880-4991","authenticated-orcid":false,"given":"Muhammad Azeem","family":"Akbar","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology Nanjing University of Aeronautics and Astronautics  Nanjing China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5786-5118","authenticated-orcid":false,"given":"Sajjad","family":"Mahmood","sequence":"additional","affiliation":[{"name":"Information and Computer Science Department King Fahd University of Petroleum and Minerals  Dhahran Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmed","family":"Alsanad","sequence":"additional","affiliation":[{"name":"STC's Artificial Intelligence Chair, Department of Information Systems, College of Computer and Information Sciences King Saud University  Riyadh Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8512-9687","authenticated-orcid":false,"given":"Abdu","family":"Gumaei","sequence":"additional","affiliation":[{"name":"STC's Artificial Intelligence Chair, Department of Information Systems, College of Computer and Information Sciences King Saud University  Riyadh Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2020,10,21]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"crossref","unstructured":"PereraP SilvaR PereraI.Improve software quality through practicing DevOps 2017 Seventeenth International Conference on Advances in ICT for Emerging Regions (ICTer) Colombo 2017:1\u20106.https:\/\/doi.org\/10.1109\/ICTER.2017.8257807","DOI":"10.1109\/ICTER.2017.8257807"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49094-6_27"},{"key":"e_1_2_10_4_1","doi-asserted-by":"crossref","unstructured":"DyckA PennersR LichterH.Towards definitions for release engineering and DevOps.2015 IEEE\/ACM 3rd International Workshop on Release Engineering. IEEE2015:3.","DOI":"10.1109\/RELENG.2015.10"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-18612-2_14"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2904354.2904372"},{"key":"e_1_2_10_7_1","first-page":"8","author":"P. Team","year":"2010","journal-title":"CMMI for development, version 1.3, improving processes for developing better products and services"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2018.032501519"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3383219.3383280"},{"key":"e_1_2_10_10_1","first-page":"436","article-title":"DORA platform: DevOps assessment and benchmarking","author":"Forsgren N","year":"2018","journal-title":"International Conference on Design Science Research in Information System and Technology, Springer, Cham"},{"key":"e_1_2_10_11_1","first-page":"600","article-title":"Composable DevOps: automated ontology\u2010based DevOps maturity analysis","author":"McCarthy MA","year":"2018","journal-title":"2015 IEEE International Conference on Services Computing, IEEE"},{"key":"e_1_2_10_12_1","unstructured":"ChrissisMB KonradM ShrumS.CMMI for development: guidelines for process integration and product improvement. Pearson Education2011."},{"key":"e_1_2_10_13_1","doi-asserted-by":"crossref","unstructured":"RafiS YuW AkbarMA.RMDevOps: a road map for improvement in DevOps activities in context of software organizations. In Proceedings of the Evaluation and Assessment in Software Engineering.2020:413\u2010418.","DOI":"10.1145\/3383219.3383278"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/Agile.2008.42"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2019.06.010"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.2263"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359981"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.86"},{"key":"e_1_2_10_19_1","first-page":"27","article-title":"Designing software architecture to support continuous delivery and DevOps: a systematic literature review","author":"Bolscher R","year":"2019","journal-title":"ICSOFT"},{"issue":"12","key":"e_1_2_10_20_1","first-page":"18","article-title":"Disciplined agile delivery and collaborative DevOps","volume":"24","author":"Ambler SW","year":"2011","journal-title":"Cutter IT J"},{"key":"e_1_2_10_21_1","first-page":"270","article-title":"A DevOps framework for quality\u2010driven self\u2010protection in web software systems","author":"Beigi\u2010Mohammadi N","year":"2018","journal-title":"Proceedings of the 28th Annual International Conference on Computer Science and Software Engineering, IBM Corp"},{"key":"e_1_2_10_22_1","first-page":"212","article-title":"Dimensions of devops","author":"Lwakatare LE","year":"2015","journal-title":"International Conference on Agile Software Development, Springer, Cham"},{"key":"e_1_2_10_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICACCE.2018.8458052"},{"key":"e_1_2_10_24_1","volume-title":"Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Adobe Reader)","author":"Humble J","year":"2010"},{"key":"e_1_2_10_25_1","doi-asserted-by":"crossref","unstructured":"KamutoMB LangermanJJ.Factors inhibiting the adoption of DevOps in large organisations: South African context. In 2017 2nd IEEE International Conference on Recent Trends in Electronics Information & Communication Technology (RTEICT) IEEE.2017:48\u201051.","DOI":"10.1109\/RTEICT.2017.8256556"},{"key":"e_1_2_10_26_1","doi-asserted-by":"crossref","unstructured":"WaseemM LiangP.Microservices architecture in DevOps. In 2017 24th Asia\u2010Pacific Software Engineering Conference Workshops (APSECW) IEEE.2017:13\u201014.","DOI":"10.1109\/APSECW.2017.18"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2018.00019"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1353-4858(18)30070-9"},{"key":"e_1_2_10_29_1","first-page":"33","article-title":"From agile development to devops: going towards faster releases at high quality\u2014experiences from an industrial context","author":"Elberzhager F","year":"2017","journal-title":"International Conference on Software Quality, Springer, Cham"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49094-6_44"},{"issue":"2","key":"e_1_2_10_31_1","first-page":"51","article-title":"DevOps shifting software engineering strategy\u2010value based perspective","volume":"17","author":"Mohamed SI","year":"2015","journal-title":"Int J Comput Eng"},{"key":"e_1_2_10_32_1","unstructured":"TeixeiraDS.Maturity model for DevOps(doctoral dissertation).2019.https:\/\/repositorio.iscte-iul.pt\/bitstream\/10071\/20297\/1\/Master_Daniel_Simoes_Teixeira.pdf"},{"key":"e_1_2_10_33_1","unstructured":"ZarourM AlhammadN AleneziM AlsarayrahK.A research on DevOps maturity models. no.2019;3:4854\u20104862."},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.03.069"},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2924542"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-73460-4_11"},{"key":"e_1_2_10_37_1","unstructured":"KhanSU.Software outsourcing vendors' readiness model (SOVRM). Ph.D. dissertation School Comput. Math. Keele Univ. Keele U.K:2011."},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297280.3297656"},{"key":"e_1_2_10_39_1","first-page":"70","article-title":"Software security in devops: synthesizing practitioners' perceptions and practices","author":"Rahman AU","year":"2016","journal-title":"2016 IEEE\/ACM International Workshop on Continuous Software Evolution and Delivery (CSED), IEEE"},{"issue":"2","key":"e_1_2_10_40_1","first-page":"81","article-title":"Chief executives de ne their own data needs","volume":"57","author":"Rockart JF","year":"1979","journal-title":"Harv Bus Rev"},{"key":"e_1_2_10_41_1","first-page":"44","article-title":"The intersection of continuous deployment and architecting process: practitioners' perspectives","author":"Shahin M","year":"2016","journal-title":"Proceedings of the 10th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, ACM"},{"key":"e_1_2_10_42_1","first-page":"111","article-title":"Beyond continuous delivery: an empirical investigation of continuous deployment challenges","author":"Shahin M","year":"2017","journal-title":"Proceedings of the 11th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, IEEE Press"},{"key":"e_1_2_10_43_1","first-page":"1","article-title":"The scope of EPIC case studies","author":"Brereton OP","year":"2007","journal-title":"EPIC technical Report EPIC"},{"key":"e_1_2_10_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2010.12.010"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.11.005"},{"key":"e_1_2_10_46_1","first-page":"1","article-title":"Towards an evidence\u2010based understanding of electronic data sources","author":"Chen L","year":"2010","journal-title":"14th International Conference on Evaluation and Assessment in Software Engineering (EASE)"},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cola.2018.12.005"},{"key":"e_1_2_10_48_1","first-page":"207","article-title":"Critical success factors for off\u2010shore software development outsourcing vendors: a systematic literature review","author":"Khan SU","year":"2009","journal-title":"Proc. IEEE Int. Conf. Global Softw. Eng. (ICGSE)"},{"key":"e_1_2_10_49_1","unstructured":"KitchenhamB ChartersS.Guidelines for performing systematic literature reviews in software engineering. Keel Univ. Keele U.K. Tech. Rep. EBSE\u20102007\u201001.2007."},{"key":"e_1_2_10_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2010.12.010"},{"key":"e_1_2_10_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.08.048"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2017.03.006"},{"key":"e_1_2_10_53_1","first-page":"17","article-title":"Systematic review of success factors for scaling agile methods in global software development environment: a client\u2010vendor perspective","author":"Shameem M","year":"2017","journal-title":"Software Engineering Conference Workshops (APSECW), IEEE"},{"key":"e_1_2_10_54_1","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2015.0038"},{"key":"e_1_2_10_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.12.005"},{"key":"e_1_2_10_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2918552"},{"key":"e_1_2_10_57_1","volume-title":"Basics of Qualitative Research: Grounded Theory Procedures and Techniques","author":"Corbin J","year":"1990"},{"key":"e_1_2_10_58_1","unstructured":"NiaziMK.A framework for assisting the design of effective implemen\u2010tation strategies for software process improvement. Ph.D. dissertation Faculty Inf. Technol. Univ. Technol. Sydney Ultimo NSW Australia;2004."},{"key":"e_1_2_10_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3210459.3210465"},{"key":"e_1_2_10_60_1","doi-asserted-by":"publisher","DOI":"10.1002\/9781118181034"},{"key":"e_1_2_10_61_1","volume-title":"Case Study Research: Design and Methods","author":"Yin RK","year":"2013"},{"key":"e_1_2_10_62_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2003.10.017"},{"key":"e_1_2_10_63_1","unstructured":"MateenA AmirH.Enhancement in the effectiveness of requirement change management model for global software development. arXiv preprint arXiv:1605.00770.2016."},{"key":"e_1_2_10_64_1","doi-asserted-by":"publisher","DOI":"10.22581\/muet1982.1803.17"},{"key":"e_1_2_10_65_1","doi-asserted-by":"publisher","DOI":"10.1201\/b17461"},{"key":"e_1_2_10_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSIT.2010.5564998"},{"key":"e_1_2_10_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICGSEW.2016.25"},{"key":"e_1_2_10_68_1","unstructured":"GarciaVC.RiSE reference model for software reuse adoption in Brazilian companies. From web sitehttp:\/\/ivanmachado.com.br\/research\/rise\/thesis\/files\/2010_ViniciusGarcia_phd.pdfaccessed on 25\/03\/2016.2010."},{"key":"e_1_2_10_69_1","unstructured":"SpoelstraW.Reusing software assets in agile development organizations\u2014a management tool. from web sitehttp:\/\/essay.utwente.nl\/59917\/1\/MA_thesis_W_Spoelstra.pdfaccessed on 30\/03\/2015.2010."},{"key":"e_1_2_10_70_1","first-page":"26","article-title":"A new reuse capability and maturity model: an overview","author":"Younoussi S","year":"2018","journal-title":"Proceedings of the 2018 International Conference on Software Engineering and Information Management, ACM"},{"key":"e_1_2_10_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/52.300079"},{"key":"e_1_2_10_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2017.2728603"},{"key":"e_1_2_10_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2840322"}],"container-title":["Journal of Software: Evolution and Process"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/smr.2323","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/smr.2323","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/smr.2323","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,29]],"date-time":"2023-08-29T16:20:00Z","timestamp":1693326000000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/smr.2323"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,21]]},"references-count":72,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["10.1002\/smr.2323"],"URL":"https:\/\/doi.org\/10.1002\/smr.2323","archive":["Portico"],"relation":{},"ISSN":["2047-7473","2047-7481"],"issn-type":[{"value":"2047-7473","type":"print"},{"value":"2047-7481","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,21]]},"assertion":[{"value":"2020-05-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-09-19","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-10-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2323"}}