Mathew Lodge, who began his career writing compilers, has over 25 years of management experience in the software industry, including Anaconda, VMware, and Weaveworks. He is currently CEO of Diffblue which offers an automated unit test writing solution, Diffblue Cover, as well as Diffblue Cover: Community Edition, a free version created for developers using IntelliJ, the most popular IDE for enterprise Java. . He has contributed to Harvard Business Review, VentureBeat and other publications.
CIOs face a strategic challenge: there is far more software to write than there are developers available to write it. Result: Software developer salaries are at an all-time high and enterprise software developer revenue was 13% between April and September 2021 according to an MIT study.
It’s not a new challenge, of course, but there’s something new on the menu: innovative solutions that give CIOs new strategic options for solving the problem.
At the highest level, there are two main approaches: finding ways to avoid writing code to solve a problem, and reducing the amount of code that must be written by developers. These fall into three main areas of intense innovation:
- No alternative code
- Low code – especially open source stacks
- AI augmented coding
CIOs in organizations focused on growth or undergoing digital transformation are uniquely positioned to adopt no-code tools: they don’t need developers and provide a level of protection against technical debt and code maintenance costs. No Code is particularly well suited for automating business processes, the essential “how work gets done” in all organizations. Manual processes – including “spreadsheet wrangling” – are growth killers because they are very fragile and break under change, and fixing them involves retraining and changing behavior.
In the past, much of this automation was implemented by software developers. Tools for enterprise programming languages such as Java followed this evolution of business automation: in the 2000s, Spring Java became the primary framework for rapidly developing web-based business applications. In the 2010s, Spring reinvented itself with Spring Boot, a framework for developing not only websites, but also APIs and the microservices that underpin them. The focus had shifted from stand-alone websites to more complex process and data integrations, to allow for a greater scope of automation (Gartner calls this the “composable enterprise”).
Yet much of what was implemented in the 2000s as a Spring Java application today could be implemented in a modern CMS from Adobe or ContentStack, and those platforms can also access data and processes behind external APIs. More general “cloud super-spreadsheets” like Airtable can assemble almost anything with an API for automation into a solid, responsive cloud-based web application.
Gartner calls this relentless automation-everywhere approach “Hyperautomation.” The opportunity for CIOs is to take advantage of the sophistication of new automation platforms so that “Business technologists” automate everything in cooperation with IT. In a case study at Heathrow Airport, IT staff established communities of interest around no-code tools to catalyze their adoption. They also established three levels of IT cooperation with the business based on scope and risk: “private pilot” (largely hands-off IT), “co-pilot” (business leaders with IT involvement ) and “passenger” (IT managers with the company). participation). A key result was avoiding £2 million in costs for traditional app development using traditional coding.
The line between no code and low code is somewhat blurred because most no-code platforms provide the ability to customize and integrate with other systems through traditional coding. And that provides a key insight: unlike no-code solutions, low-code tools are used by people who write code for a living. They may not have a developer title, but they write code and are more expensive and rarer – data scientists are a good example – but not as expensive and rare as professional developers.
Low-code platforms are nothing new: OutSystems, one of the biggest players in the industry, was founded 21 years ago. What’s new is low code based on open source stacks. CIOs can give their organizations access to a much larger pool of open source low coders and also make their organizations a more attractive place to work. Make the cool factor of open source work for you!
Data science is a good example: Python is the go-to language for data science because it’s free and easy to learn, and thanks to the excellent ecosystem of open source data science libraries. NumPy, the basic scientific Python math library, is found in the dependency tree of almost all Python data science programs and libraries. Can you do data science in Java and code what NumPy does? Sure. Is it as simple as doing it in Python? Not even close. Do Data Scientists Want to Write in Java? No.
Since Python data science programs run on a thick stack of open source libraries that do the heavy lifting, a small amount of Python data science code can do a lot. A few lines of Python can load data and start scalable parallel training for a neural network. Notebook technology – primarily the open source Jupyter Notebook – makes mixing code, data, text, graphics, and images as easy as writing a web page, without writing any additional code.
Open source low-code stacks help CIOs because they require less code to write and they don’t have to be written by professional developers. There is also a large ecosystem of vendors who can guarantee, secure and support popular open source stacks and will gladly sell you platforms that make them easier to manage, govern and use.
AI augmented coding
One of the reasons there is more code to write than people to write it is the often overlooked fact that 99.999% of code is written by hand. Software has transformed productivity in many areas – you couldn’t start a new business in most areas where 99.999% of the work was done by hand. But that’s what we do with software. It’s like the ideas and approaches of the industrial revolution and everything that followed never touched software writing itself.
To be fair to developers, previous attempts at automated code generation were crude and inefficient, generating clunky code that only a machine could love but which, ironically, human developers would be required to maintain. Unsurprisingly, they didn’t want to.
What’s new is the application of AI and machine learning techniques, which has opened the door to human-like code. Today, there are two main approaches to AI-augmented coding: automated code completion tools and automated test writing tools.
AI-based code competition tools are a big step up from basic code completion present in current IDEs: they write more substantial chunks of code trying to infer the developer’s intent from the code and/or comments that preceded. These tools grew out of work on Transformers, a form of neural network designed for natural language processing – language translation, text summarization and similar applications. GitHub’s CoPilot is perhaps the best known and is currently in technical preview.
Transformers are trained on a very large corpus of examples and they use the resulting model to synthesize new examples – in the case of CoPilot, likely completions of code written so far.
According to GitHub, CoPilot achieves this about 43% of the time, so developers should review the code generated by the tool. It’s more useful than it sounds – a strength of CoPilot is generating code that calls “foreign lands” such as cloud service APIs. To write this type of code, most developers read the documentation, then run Google or Stack Overflow searches to find sample code they can adapt, then test them in a trial-and-error process. CoPilot writes a complete call in seconds, which the developer can try immediately. It’s also good for writing “standard code” – the type of standardized code that the language or framework requires to be populated to form the basic code structure.
Automated test writing tools use reinforcement learning to write unit tests that match the code they test, resulting in a test suite that can find regressions in subsequent code changes. This is especially useful for poorly tested existing codebases and legacy code being modernized.
They work by searching for the best test that achieves the best code coverage. They write a test and see how well it works, and based on that predict a better test, try that, and so on. Good test results strengthen the direction the algorithm moves through the set of all possible tests, hence its name. Although lesser known than neural networks, reinforcement learning is growing in popularity and forms the algorithmic heart of some spectacular AI success stories, such as Google AlphaGo.
Diffblue, an AI spin-off from the University of Oxford, offers a product that writes and maintains 100% autonomous Java unit tests using reinforcement learning. It is used by major banks, pension funds, and healthcare companies to write and maintain unit tests for its transformers and reinforcement learning.
Characteristic picture Going through Pixabay.