The Greatest Challenges Facing Software Engineering in the Next 20 Years

Software Engineering is a very important field of study regarding software development since the beginning of modern computing and the rise of software complexity.

If you consider that to achieve a deliverable software component you may need to understand more than just bits and bytes, moreover, for software to be successful in the piece of the problem it may solve, there are much more than just electrical circuits behind that.

Based on that affirmation you should consider that software development requires more than machines and code, but also, the idea of the business behind it to be understood, the abstraction while designing based on its requirements, the right methodologies and procedures which will make the code with quality enough to support maintenance, the testing environment which will confirm that the software is reliable enough to run and automate some human task.

In the enumeration below I list my personal view of the greatest challenges regarding software engineering in the next 20 years.

#1 Testing multiple software components with integration with other software components

I personally believe one of the most critical software development phases is the integration testing. If we consider that today we have more and more integrated applications running in a space where many applications depend on each other, how to guarantee our software will work integrated with other software components we do not know much about its infrastructure and availability?

Moreover, according to StackOverflow, 2017, there’s the issue of data that changes in back-end systems over time. Sometimes you don’t really have any influence over these changes, and they can render detailed integration tests useless. Often this results in making “smoke test” like integration tests that really don’t assert too much of the content of the back-end systems.

I believe in a twenty year period, more applications will be connected and more will be integrated and dependent of each other, so how to make integration testing with all these applications and how to guarantee all them will work in a 24/7 scheme?

#2 Accuracy in Software Development Cost Estimates

According to Fowler, 2003: “One of my biggest irritations are studies of productivity based on lines of code. For a start, there’s all the stuff about differences between languages, different counting styles, and differences due to formatting conventions. But even if you use a consistent counting standard on programs in the same language, all auto-formatted to a single style – lines of code still doesn’t measure output properly.”

As long as more modern software development tools are created and developed, one thing I personally believe is that it will be possible to be more accurate in terms of cost and productivity estimates, which with my experience in the software development area it is strictly difficult to measure cost and productivity estimates with accuracy, and the way more specific development tools are available in order to provide more ways to be accurate with estimates and to measure productivity, but this is a constant challenge the science of software engineering is facing and will continue to face during the next years.


Martin Fowler. 2003. CannotMeasureProductivity. [ONLINE] Available at: [Accessed 1 June 2017].

Stackoverflow. 2017. What are the challenges you have faced in writing integration tests?. [ONLINE] Available at: [Accessed 1 June 2017].

Real Time Systems


A RTS or “Real Time System” is an information processing system which responds to external command inputs, and delivers information in a reliable expected time (Petters, 2008). It is very common to be embedded systems running inside cars, electronic devices, airplanes and cell phones, for instance.


One of the most important subjects in this discussion is the timing non-functional requirement for real-time systems. On the need of responding in expected time, the timing should flow the way as the requirement expects, in order to do not have the loss in the quality of the real-time system.

According to Magalhães (1993), real-time systems can be cataloged into two views:

  • Soft real-time System

    • Soft real-time systems are systems in which not meeting a timing deadline will not affect or damage the environment, which makes this acceptable.

  • Hard real-time System

    • A hard real-time system, it would not be acceptable that a timing deadline to be not met because it would have catastrophic results to the environment.


The most difficult concern and OOP and real-time systems in the concurrency because of the need for fusing the two concepts, which are different. According to (R. Welch, L. & Dieter K., H, 1996) the difficulties while working with OOP programming languages to build RTS’s are:

#1 Parallelization in real-time systems should not be arbitrary but according to the application semantics;

#2 Traditional real-time computing paradigms may not be adequate in addition to objects, abstractions at higher levels of granularity are needed (e.g., execution paths consisting of time-constrained sequences of method invocations);

#3 Object-oriented features that make real-time system engineering difficult include: method lookup, garbage;

#4 Collection, and variation in invocation latency due to object distribution and dynamic binding;


In spite of the object-oriented paradigm to be very popular in the development of enterprise applications, the low-level programming languages such as C (for instance) are still a better fit for programming real-time systems. The ‘virtual world’ created by the virtual machines in order to allow programmers to deal with more ‘human’ data types and classes/objects allows more ease while developing some kinds of systems, but systems that get closer of the machine and low-level requirements, usually tends to be best solved with low-level programming languages.


Petters, S., 2008. Real-Time Systems Lecture, [PPT] Australia: UNSW SIDNEY. Available at: <; [Accessed 23 July 2017].

António P. Magalhães, et. al, 1993. Deadlines in Real-Time Systems, [Technical Report] Portugal: Faculdade de Ciências e Tecnologia, Universidade de Ciombra (Portugal). Available at: <; [Accessed 23 July 2017].

R. Welch, L. & Dieter K., H., 1996. OOPS Messenger special issue on object-oriented real-time systems. ACM SIGPLAN, 1 January, pp.1-2.

The Service Oriented Architecture


The objective of this discussion is to introduce SOA, to provide a view of how software can be integrated and developed in such manner to allow software solutions to be interconnected, the problems of this approach, a suggestion of solutions and at the end a conclusion.


The SOA – Service Oriented Architecture was born based on the need of the legacy software systems to get interconnected. You can imagine a bunch of many systems developed during many years in different locations inside a company, what leads to a high granular reality in the company software space (and you should consider all that applications built in many different languages and technologies) (Abrams, 2008).

SOA is based on low coupled services, which can be written in different technologies or programming languages, what makes the integration between solutions in an interoperable reality (Kodali, 2005).


In the list below, it is enumerated some of the common problems on the attempts on integration with SOA.

#1 DIFFERENT DOMAINS: Based on my experience, most systems do not capture the reality of the business equally, so for one system a client can be designed simply as a client but for another system, it was designed as a company. Different languages in SOA leads to the use of the Enterprise Service Bus translators.

#2 SECURITY CONCERNS: While there are OASIS standards to implement security with SOA Webservices (Oasis, 2005), the need of transmitting data may lead to security interception of data or even a man-in-the-middle attack, something can compromise the whole company and data.

#3 DESIGN FOR PERFORMANCE: A bunch of XML traffic would not have its better performance while working with SOAP standards.


#1 DIFFERENT DOMAINS: Different languages in SOA leads to the use of the Enterprise Service Bus translators, it may have the mapping of from-to, which will allow the integration to be done, however when designing new solutions, I would suggest using common libraries with common domain classes, which represents the whole company domain and to be shared on the company projects.

#2 SECURITY CONCERNS: The use of HTTPS joined together with WS-security (Oasis, 2005), would allow more security to the system and securing the man-in-the-middle attacks.

#3 DESIGN FOR PERFORMANCE: A REST approach would best fit for solutions which needs more performance with the use of resources of the HTTP protocol (Roy, 2000). The use of JavaScript Object Notation (JSON) as a protocol would allow the use of less network traffic.


The use of the Service Oriented Architecture design pattern allows system integration, in a world which connection is a key concern, removing all that spreadsheet and importation integration used in the past.

SOA became a standard in the market for years, now a new design pattern is emerging to solve most of the problems SOA had in the past, which is called the microservices architectural design pattern.


Charles Abrams, R. W. S., 2008. Service-Oriented Architecture Overview and Guide to SOA Research, [Electronic Article] USA: Gartner. Available at: [Accessed 16 July 2017].

Kodali, R. R., 2005. What is service-oriented architecture?. [Online] Available at: [Accessed 16 July 2017].

OASIS, 2005. Web Services Security UsernameToken Profile 1.1. [Online] Available at: [Accessed 16 July 2017].

Thomas Fielding, Roy, 2000. Architectural Styles and the Design of Network-based Software Architectures. Doctor. California: UC Irvine.

Software Reuse


In computer software development, it is not common that a project will be completed developed since the beginning, with its developers writing every line of code and coming up with this complete solution.

Since the first modern programming language, there is the possibility of importing and including code from other developers, something very common while developing computer software.

For instance, if the programmer is writing software to download a file from another computer in an internet server, this programmer will not have the need of writing all the low-level structures to access the network card, dealing with sockets or dealing with the HTTP protocol, since network drivers are installed on the operating system, sockets are already implemented by a provided API, and the HTTP protocol has thousands of implementations available.

Even the programming languages like C, Ada, Lisp, Smalltalk, and ML where not treated as examples of software re-use in the literature, however, the goals and achievements for high-level languages have strong parallels to the current-day aspirations of software reuse researchers (Krueger, 1992).


According to the Lombard Hill Group (2017), some of the benefits of the software reuse are listed below:

  • Increase productivity: software re-use will allow the productivity to increase since the engineers will not have to “re-invent the wheel” and reduce costs in the product development;

  • Shorten-time-to-market: with the reuse of software companies have reported a short time to market, something enables better competitivity;

  • Improve software quality: software that has been used multiple times will have a big probability of having less failure than fresh developed software;


The software reuse is very common and associated with the free software or open source ideology. Richard Stallman founded in 1985, the Free Software Foundation which has a philosophy of allowing developers to copy, redistribute or modify libraries and software of other people. (Stallman, 1985)

Richard Stallman, makes an analogy of the free software philosophy in the Linux Documentary entitled The Code where he explains that

A program is a lot like a recipe, each of one has steps to be carried out, with rules which tells you when you are done or when to go back and at the end, there is a certain result. If you cook, you probably exchange recipes with your friends, and you probably change recipes too, and if you made changes and you like the result, then you might give your friends the change of the recipe. So, imagine a world where you can’t change a recipe because somebody went gone out of his way to set it up that it is impossible to change it and imagine that if you share the recipe with your friends, they will call you a pirate and try to put you in prison for years (The Code Documentary, 2001)


W. KRUEGER, C, 1992. Software Reuse. ACM Computing Surveys (CSUR), [Online]. Vol 4. Issue 2., 131-183. Available at: [Accessed 2 July 2017].

What is Software Reuse?. 2017. Lombard Hill Group. [ONLINE] Available at: [Accessed 2 July 2017].

Richard Stallman. 1985. Free Software Foundation. [ONLINE] Available at: [Accessed 2 July 2017].

Hannu Puttonen. (2001). The Code Documentary. [Online Video]. 2001. Available from: [Accessed: 2 July 2017].

The Negative Impacts Caused by Software Projects Risk


Project planning is an essential task in software development projects. According to Sommerville (2007), good management cannot guarantee the success of a software project, but bad management usually reflects in the failure of the projects. In the software project management Risk Analysis is a key subject to be included in the project plan.

As a job of project managers, identifying and documenting the risks will allow the whole development team to take actions to mitigate it to do not let the project to fail.

As seen in the picture below, project risks should be identified, analyzed, planned and monitored in order to allow the actions to mitigate them to do not let the project to have problems, which will delay or even make it fail.

riskstableFigure 1 The risk management process

My discussion relates below the three risks a project may have and how they can lead to negative impacts on a software project.


Scope creep can be defined as anything which is added to the project and was not part of the initial scope or plan. According to Kendrick (2015), by adding little value, the project will get more and more delayed, and the changes in scope delay so much that the ultimate deliverable has no value anymore because the need will no longer press or met by other means. Scope creep may lead a project to real negative impacts, and this is a very common subject in the project planning.


Based on my experience, since the beginning of the Project, after gathering the non-functional and even functional requirements, a good architecture solution must be designed. If you consider the Java environment, for instance, you may have different approaches to work with web applications. Component-based approaches may lead to a better user experience, depending on the solution it is required to be met, using component-based approaches may lead to performance issues in the user browser and even in the application server, on the other hand going with action controller based applications, it may lead to more performance, however less productivity.

Less performance may lead to a project to fail if we consider a platform which allows millions of users to be online at the same time. Less productivity also may lead to the project to fail, if the project-budget is limited.


If we consider a software process like RUP, for instance, where most of the artifacts keep the project information, an often company turn-over would not let a project to have more than 80% of having loss, however if we consider a project where the agile methodologies are being used, the company turn-over may lead the project to have many losses or even to fail, and I’d say in more than 80% of chances. The weak documentation or even the lack of resources to understand the project would make the new stakeholders to a big delay in terms of deliveries and the outcome might not meet the software initial requirements.


Sommerville, I, 2007. Software Engineering. 8th. England: Pearson. pp. 104-113.

Kendrick, T, 2015. Identifying and Managing Project Risk. 1st. New York: Amacom. pp. 52-53.

The Logical View in Software Architecture


The 4+1 view model was designed to describe the system in different views, for different stakeholders, being separated in ‘logical’, ‘development’, ‘process’ and ‘physical’ views, which helps in graphically representing and describing the architectural solution designed (Kruchten, 1995). The idea of the 4+1 view model was to come up with a representational graphical manner to separate the different aspects of a software system into different views of the system.


The 4+1 view model proposed in 1995 suggests that the software architecture is represented in 5 segregated views, which are commented below (Staveley, 2011):

  • The logical view: Demonstrating the object model of the design (e.g Class Diagram, Sequence Diagram, State machine Diagrams);

  • The process view: Refers to the synchronization and concurrency aspects of a solution design (e.g Activity Diagrams);

  • The physical view: Refers to the deployment aspects, such as how many nodes will be used and what will be deployed in that node (e.g Deployment Diagram);

  • The development view: Refers to the software modules and subsystems (e.g Package and component diagrams);

  • Use case view: Refers to the functionality of the system (e.g Use Case Diagram).


The Unified Modelling Language (UML) came up after 2 years of the 4+1 view model and today it is a standard in software development to represent graphically object-oriented software development complexity to make understandable the technical aspects of a designed solution.

The main objective of this small article is to introduce the sequence diagram which may be cataloged in the logical view of the 4+1 view model, discuss some of the advantages of using this diagram and disadvantages.


The sequence diagram represents an exemplary snippet of a software system process (B. Rumpe, 2016). The communication between objects, the system flow, and even parameters in object messages can be represented through the sequence diagram.


Figure 1 An example of a sequence diagram

The representation of the diagram above, simply describes the web process of an Actor call a findStudent() method, going through many layers of an imaginary software and returning the result back.

Based on my experience, when a software solution design contemplates a sequence diagram, it is much easier to follow the flow proposed in the architecture, but there is a point very important to bring up: the more you have in the UML documentation the easier will be to a developer understand what is meant to be done. Considering UML not only for documentation but for development, If you consider distributed teams geographically dispersed, as much as you have in documentation using graphical representation like the UML sequence diagram, the easier will be to the developers to come up with the right solution.


Kruchten, P, 1995. Architectural Blueprints—The “4+1” View Model of Software Architecture. IEEE Software 12, [Online]. 6, 42-50. Available at: [Accessed 18 June 2017].

Alex Staveley. 2011. The “4+1” View Model of Software Architecture. [ONLINE] Available at: [Accessed 18 June 2017].

Rumpe, B, 2016. Modeling with UML. 1st ed. Switzerland: Springer International.

Socio-technical Systems in a Nutshell


A system can be defined as a collection of components working together to achieve a common objective (Sommerville, 2013). People as well interact with technology in order to achieve some goal in organizations bringing up the concept of socio-technical systems, which is basically technological resources which help people to interact themselves and organizations to achieve their goals.

According to the Interaction Design Foundation, 2017, these below are examples of socio-technical (STS) systems:

  • E-mail;

  • Twitter;

  • Facebook;

  • YouTube;

  • Chat;

The technology we have today allows us to have more friendly contact with our family, friends, coworkers, and people involved in our lives. If we consider today we can chat live with many people in different localizations and in a high-resolution online meeting via Skype, for instance, we can see the impact of technology in the people interaction.

The way we work today in multi-cultural companies, especially in multinational companies are much more dynamic and the impact of the socio-technical systems, allows this kind of not only communication improvements but also of collaborative work to flow better, the way tools helps a lot this kind of activity.


System failure may occur in any system because of hardware or even software issues. If we consider that systems should be tolerant to failure because of the software engineering methods to do not be developed enough to produce systems with no-failure.

In the past, when systems used to be written in C and C++, the probability of a system to produce failure was far beyond higher than the technology it is written today in the Java and .NET, which are more modern platforms. The introduction of the high-level programming languages allowed developers to focus more on the software to be developed based on the business solution they want to achieve and focus less on the low-level machine instructions and memory management issues in which may lead to system failures.


Software products are usually developed as projects in software house companies, which may have a start and an end. Software Engineers in order to produce software systems, are based on the development of the software engineering and project management studies, which are understood and allowed by the research distributed and shared all over the world based on the experience acquired by all the individuals which may share it in masters or doctoral studies like this one we have at the University of Liverpool Online.

The impact on socio-technical systems comes out when software needs changes and upgrades because considering we have changes in the business operations and changes caused by technological advances, for instance, it all comes to planning in which is responsible to continuously improve the software based on its needs.


Ian Sommerville. (2013). Introducing sociotechnical systems. [Online Video]. 23 November 2013. Available from: [Accessed: 14 June 2017].

Interaction Design Foundation. 2017. The Encyclopedia of Human-Computer Interaction, 2nd Ed.. [ONLINE] Available at: [Accessed 15 June 2017].

The Advantages and Disadvantages of Using Agile Methods

The agile culture in the software engineering science started with the Agile Manifesto (Cunningham et al., 2001), written in 2001 by seventeen visionaries in which where working to find new ways of developing better software, which where also brought some principles commented based on my point of view and listed in the enumeration below:

  • Individuals and interactions over processes and tools: Most software engineering processes started to produce thousands of pages and design models, diagrams, process workflow, etc… This principle changes dramatically the way things are organized in software development;

  • Working software over comprehensive documentation: Instead of writing a lot of comprehensive documentation, this principle put the idea of writing deliverable software instead of documentation first or to do not have documentation – it depends on the project;

  • Customer collaboration over contract negotiation: Tends to have the customer involved in your process and not only and simply negotiating contracts; Iterative incremental approaches helps on that way to produce with better software quality.

  • Responding to change over following a plan: Iterations in the software development process may answer to changes and not follow a plan. Moreover, with the introduction of the concept of DevOps and continuous integration, for instance, you see that the agile principles written in the agile manifesto are pertinent until today.

Based on those principles, many processes tools and approaches where being produced over all these years in order to improve software development approaches. In the following items, I enumerate some of the advantages and disadvantages of using the agile methodology, considering processes, delivery, maintenance, test automation, and software documentation.


Faster deliveries: With the implementation of the agile culture in terms on software process in software development, it can be observed that most of the software developed with iterative incremental approaches such as Scrum leads to faster deliveries, since based on its principles, Scrum, for instance, works with a prioritized product backlog, in which may lead to a faster impression on the results of the project, which also may lead to a big change in the project.

Agile testing culture: The agile culture brought to us really good software development techniques such as TDD – Test Driven Development, in which allows to build testable software components, and more than that: to build software components with the test in mind leading to better software architectures(Clifton, 2014) and afterwards to be put in a continuous integration approach leading to regression testing which leads to a good practice looking over to keep the product integrity. Based on my experience, it can be only achieved with the agile culture in projects.


Difficult to work with big teams: Based on my experience, agile methods work better when you have a smaller group of people working together. To be more precise, if you compare to RUP, where all the requirements and guidance to develop the software is written within specifications, you see that working with bigger groups of people may have a better result with ‘standard’ software development processes.

Weak documentation: In spite of not being a principle of not producing documentation, but to put deliverable in bigger precedence, seems like there is a lack of documentation while working with agile methodologies, because of its dynamic process. Documenting requirements, interactions and workflows after having the product may seem like no sense in the agile culture.

Continuous UAT: The need for continuously validating with User Acceptance Tests, may lead to project cost to increase, in spite of possibly leading to better final software quality (Waters, 2007).


Cunningham et al. 2017. Manifesto for Agile Software Development. [ONLINE] Available at: [Accessed 11 June 2017].

Kelly Waters. 2007. Disadvantages of Agile Development. [ONLINE] Available at: [Accessed 11 June 2017].

Marc Clifton. 2014. TDD is not the Failure, our Culture of Development Is. [ONLINE] Available at: [Accessed 11 June 2017].

Bill Gates Letter to Hobbyists

Today I am here to share with you something very interesting about what happened in the ’70s when the code was shared by hobbyists which used to develop software in the universities.

I just knew this after watching the movie Revolution OS (which is very nice indeed), and the purpose of this post is to show you who was the “enterprising” mind behind corporate software: Bill Gates — and I just don’t want to make the idea I am against him, of course not, since I believe he did a lot for computer science and Microsoft looks like is changing their minds to open source software since cloud computing took the scene.

There it follows the e-mail Bill Gates sent to the hobbyists encouraging them to make money with software and closing codes/patents — whatever:


p.s:  A very interesting movie to watch is the Revolution OS — a movie about Linux, which tells a bit about Linux, GNU the all the Open Source Software and initiatives. You can find it on

Deadlock from Multithreading


The objective of this small article is to give an introduction about threads, deadlocks and how a deadlock may occur illustrated with code using the Java programming language. In the end, I give my personal point of view about the subject.


A thread can be defined simply by “a separated computation process” that may be children of other threads – or the main program (Savitch, 2010). Threads can also operate in a parallel manner, what means multiple computation processes sharing the same resources, and running for a common goal.

In Java, this can be achieved by using the Thread class, which offers support for implementing single and multithread applications.

deadlockFigure 1 Illustration of a Thread Program in Java

The code figure above shows a simple thread program coded up with the Java language. Basically, this illustrates one thread instanced by the Thread class. The ThreadExample1 class is just an implementation of Runnable, one of the ways of implementing threads with the Java programming language.


Figure 2 Example of a Multithreading application in Java

When running the code above, will give the output of:



Is off


Is on

Is off

Is on

Is off

Is on

Is off


Table 1 Output of the program


Deadlock is defined by, 2016 as A deadlock is a situation in which two computer programs sharing the same resource are effectively preventing each other from accessing the resource, resulting in both programs ceasing to function.

Figure 3 An illustration of a dead lock

The figure above illustrates a deadlock occurring on the traffic. What happens is that all the cars are trying to access the same resource at the same time, what locks the traffic to flow.


“When in doubt, synchronize.”

(Goetz, 2001)

The Java programming language offers synchronization resources in order to prevent deadlocks. What happens is that at every variable that is overwritten using multiple threads, it is strictly recommended that you use synchronization.

Example of a potential synchronization deadlock:


Figure 4 Using the synchronization resource (Goetz, 2001)

Image a scenario while thread A calls oneMethod(), while thread B calls anotherMethod(). What happens is that thread A gets the lock on cacheLock and at the same time thread b gets the lock on tableLock. Now the threads are deadlocked.


According to the StackOverflow, 2016 forum, some good tips to avoid deadlocks are:

  • Always use lock-free data-structures;

  • Release locks after a timeout period;

  • Always acquire the locks in the same order.


Deadlocks are a very sensible threat to every program which uses multithreading programs sharing the same resources. While debugging and simulating deadlocks are difficult because it needs timing and also it may not occur because of the environment and machine load. The Java programming language offers resources to deal with deadlock issues, however, it is more of a programmers responsibility to avoid this problem.


Savitch, W, 2013. Absolute Java. 5th ed. USA: Pearson. 2016. What is deadlock? – Definition from [ONLINE] Available at: [Accessed 01 May 2016].

Blogspot. 2016. A figure of a deadlock. [ONLINE] Available at: [Accessed 1 May 2016].

Brian Goetz. 2001. Avoid synchronization deadlocks | JavaWorld. [ONLINE] Available at: [Accessed 01 May 2016].

Multithreading – Tips to prevent deadlocks in java. 2016. Stack Overflow. [ONLINE] Available at: [Accessed 01 May 2016].