Enterprise Architecture with .NET E-Book

Jean-Philippe Gouigoux

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 1177

Veröffentlichungsjahr: 2024

Beschreibung

The software development domain continues to grow exponentially, and information systems have become the backbone of most industries, including non-digital-native ones. However, technical debt, coupling, and a high level of maintenance - sometimes bringing IT systems to a complete halt – continue to present a problem. The software industry has to still apply standards-based, modular, and repeatable approaches that exist in other industries.
This book demonstrates such methods in action, particularly business/IT alignment principles. As you progress, you’ll cover advanced concepts and theories currently researched in academia. Then, you’ll be guided toward a practical framework to transfer these approaches to actual software architecture. Finally, a dedicated section will help you apply the knowledge you gain to a sample application in .NET where API design, dependency management, and code writing will be explained in detail to relate to the business-alignment principles explained at the beginning. Throughout the book, you’ll get equipped with the skills to create modular, long-living applications that serve your users better.
By the end of this .NET book, you’ll not only have learned new concepts but also gained the ability to apply them immediately to your upcoming software endeavors.

Leseprobe

Enterprise Architecture with .NET

Expert-backed advice for information system design, down to .NET and C# implementation

Jean-Philippe Gouigoux

Enterprise Architecture with .NET

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Kunal Sawant

Publishing Product Manager: Teny Thomas

Senior Content Development Editor: Rosal Colaco

Book Project Manager: Deeksha Thakkar

Technical Editor: Vidhisha Patidar

Copy Editor: Safis Editing

Indexer: Hemangini Bari

Production Designer: Joshua Misquitta

DevRel Marketing Coordinator: Shrinidhi Manoharan

First published: May 2024

Production reference: 1170524

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83508-566-0

www.packtpub.com

To the memory of my father, Jacques, who died just before reaching his magic age of 88. Thanks for teaching me what industry means, both in terms of hard work and in doing things the right way; you inspired a lot of this book.

– Jean-Philippe Gouigoux

Foreword

Any complex organization, whether it’s a service company or an industry, requires rigorous management, made even more complex due to its intertwined various facets. Facing digital transformation pitfalls while increasing performance and productivity needs to be conducted as a whole, from strategic to operational and technical levels, involving working with a systemic approach thanks to the information system field.

I have known Jean-Philippe Gouigoux for more than 10 years. Both passionate about the challenges posed by the digital transformation of organizations, we have combined our respective academic and business expertise to offer formalized feedback and recommendations through international scientific publications on the migration of legacy systems to microservices-based architectures and, more recently, on business-IT alignment issues. The main lesson learned is that business needs and best practices must always drive technological changes. In his book, Jean-Philippe, thanks to his extensive technical experience as an architect and manager and as an author and consultant, provides valuable advice on how to design viable information systems. The relevance of his book is that it covers the business aspects of the information system and, at the same time, an enlightened and practical introduction to software architecture and application aspects in .NET and their deployment.

This book is a goldmine for all developers who want to see beyond code and for all architects who want to see beyond software architectures, to work toward the achievement of their organization’s success by creating efficient and sustainable information systems leaning on business-aligned services.

Dalila Tamzalit

Senior Associate-Professor, Nantes University. PhD. Habilitation to Direct Research.

Contributors

About the author

Jean-Philippe Gouigoux started programming in 1985 and has never stopped since then, working as a developer, then an architect, an R&D manager, and today, the chief technical officer of a group of software editors. Coming from a mechanical engineering background, his main career goal is to apply a rigorous approach to information technology and try to make information systems as modular and standardized as mechanical systems. JP has written more than 10 technical books and has spoken at around 100 conferences and on university courses, teaching architecture, testing, service-oriented design, performance in .NET, Docker and Kubernetes, and so on. He has been a Microsoft MVP since 2011.

Big thanks to Guénaëlle, Clémence, and Mathieu for supporting my dedication to writing. My motivation has always been to make you proud.

About the reviewer

Guillaume Collic is a passionate software developer who loves software fundamentals, the craftsmanship movement, functional programming, Domain-Driven Design (DDD), and all approaches that help make the domain clearer and IT aligned with business. He was a Microsoft MVP from 2013 to 2019. He currently works as a staff engineer at Agicap. When he’s not working, he spends time coding and sharing with communities (board game enthusiasts, makers, developers...).

Table of Contents

Preface

Part 1: Business-Aligned Architecture and the Problems It Solves

1

The Sad State of Information Systems

What is an information system?

A quick history of information systems

Software building – still craftsmanship

Craftsmanship as opposed to previous lack of quality

A word about “emerging architecture”

Craftsmanship as opposed to an industrialized approach

Concept of technical debt

The long-standing comparison of software to mechanical systems

The efficiency of an IS

The measure of efficiency of a system

The cost of maintenance

Examples of the prohibited evolution of IT systems

Classification of the causes

The classical symptoms of a blocked IS

Rate of renewal of large systems

Summary

Further reading

2

Applying Industrial Principles to Software

What is an industry?

The two roots of industrialization – modularization and standardization

Modularity to reduce complexity

Standardization to ensure modularity is helpful

Management of complexity

Different types of complexity

Computer science as a way to deal with complexity

Information systems and complexity

The concept of “as a service”

Link to a minimum viable product

The benefits of standards and norms

Docker, containers, and OCI

Another example with IAM

The urbanism metaphor of information systems

Summary

3

Reaching Business Alignment

Technical requirements

Software for business and the principle of alignment

The jungle of technical recommendations

An example with KISS, DRY, and WET

The particular case of internal toolboxes/frameworks

The only stable guide: aligning to your users’ business

Digression on the digital transformation of companies

Conway’s law applied to application and systems

Introducing the CIGREF diagram

Process level

Functional level

Software layer

Hardware layer

Using the four-layer diagram

What should we map?

How to start drawing a four-layer diagram

A generic difficulty when aligning information systems

Evolving the information system in time

The four-layer diagram method for service providers

Patterns and antipatterns of alignment

The sad reality of alignment

What we can aim for

The main antipatterns in alignment

Some other antipatterns and a proposed classification

Summary

4

Dealing with Time and Technical Debt

The impact of time on systems due to functional change

A bit of fun with ill-placed comparisons

Consequences in the software world

How the Agile approach aims at solving the time problem

A metaphor to explain Agile

Back to the concept of emerging architecture

Apparent opposition between architecture and Agile methods

The position of famous architects

Thinking ahead with function contracts

The concept of technical debt

The general definition of technical debt

Causes of technical debt and its relation to time

Technical debt’s relation to time

Debt or usury?

Ways to balance technical debt

The Big Bang temptation

An experience-proven blueprint method for information systems

It all starts with... mapping!

Finding the atomic actions

A few last words on time, semantics, and alignment

Summary

5

A Utopic Perfect IT System

The concept of an ideal IT system

The use for an ideal structure

The origin of the design

Using a vision to define a target

Data management in the ideal system

Data is the blood of information systems

Data as the 21st-century oil – really?

A really “know it all” system

Relationship with CQRS

The need for data quality

Designing a generic “referential”

Choices of implementation for an MDM

Rules management in the ideal system

Rules as the nervous network of the information system

BRMS as a dedicated tool for business rule handling

Common implementations of business rules

Processes management in the ideal system

Processes as the brain of the information system

Implementations of a BPM

How close can we get to this utopic system?

Favoring contexts

Approaching the target, the Agile way

Summary

Further reading

Part 2: Architecture Frameworks and Methods

6

SOLID Principles, from Code to Systems

Describing the sample information system requirements

The company and its business

The users and actors of the information system

Data manipulation

The stakes of the information system for the company

The situation as seen by the person put in charge of improving the system

SOLID principles and application to systems of whatever scale

Single responsibility principle

Open/closed principle

Liskov substitution principle

Interface segregation principle

Dependency inversion approach

Critical analysis of SOLID

Limits of the separation of responsibilities

About the polemics on monoliths

Beware of unintended coupling

Wrapping up before going back to the demo

7

C4 and Other Approaches

The C4 approach

The context level

The container level

The component level

The code level

Clean architecture, hexagonal architecture, and other frameworks

Technical architecture patterns

Onion architecture

Hexagonal architecture

Clean architecture

Summary

8

Service Orientation and APIs

Looking at the history of service orientation

The long-awaited reusability

Routines and avoiding punching additional paper cards

Libraries and the capacity to share instructions between programs

Attempts at general interoperability

Using web standards to try and get universal

The steps in middleware

The most recent evolution – REST APIs

What we are missing to reach actual service reusability

Characteristics of a service

As a service explained

Getting rid of middleware altogether

External interop finally becoming a reality

Interop made real with standards

Keeping complete compatibility

Managing APIs

Inversion of dependency for services

Application to our demonstration system

Interfaces needing analysis

Using norms and standards

Where do I find norms and standards?

Pivotal format for the other interfaces

Summary

9

Exploring Domain-Driven Design and Semantics

A functional approach to a functional problem

The importance of semantics

DDD

Application to clean information system architecture

Using entities in referential applications

Managing the life cycle of entities

Relation between subdomains and time

Link to the API and services

Including time in an API

Aligning an API to subdomains and the consequences

API testability

Summary

10

Master Data Management

Responsibilities around the data

Persistence

The question of identifiers

The single entity reading responsibility

Other kinds of reading responsibilities

The complex art of deleting data

Relation to specialized links

So-called secondary features

Metadata and special kinds of data

The different kinds of data referential applications

Centralized architecture

Clone architecture

Consolidated and distributed architectures

Other types of referential architectures

Some architectural choices

Tabular versus NoSQL

CQRS and event sourcing

One more step beyond databases – storing in memory

Patterns of data evolution in time

Data governance

Progressive implementation of a unique referential

Keeping an eye on dogma and useless complexity

Summary

11

Business Processes and Low Code

Business processes and the BPMN approach

What is a business process?

Granularity in the processes

Limits of the processes

Business process modeling

The history of BPM standards

The BPMN 2.0 standard

Business processes software-based execution

Principles

Application to our example information system

Link to users

Available software for BPMN edition and execution

Why so little use of BPMN in the industry?

Other associated practices

Business activity monitoring

Business process simulation and optimization

Business process mining

Other approaches to business process implementation

Process in the GUI

Process in higher-level APIs

Process in a MOM/ESB

Low-code/no-code approaches to processes

Choreography instead of orchestration

Implementing choreography

Should I use BPMN in my information system?

Summary

12

Externalization of Business Rules

Business Rules Management Systems

How does a BRMS handle business rules?

Additional characteristics of a BRMS

Actual use of BRMS

Examples of a BRMS

The DMN standard

The origin and principle of DMN

Implementations

An example of DMN use

Calling the business rule runtime

Summary

13

Externalization of Authorization

A BRMS and authorization management

The semantics of identity and authorization management

A digression on IAM implementation

Role-based access control and attribute-based access control models

The XACML approach

A digression on the granularity of microservices

Applying authorization to our sample information system

The Open Policy Agent alternative

The functional needs of DemoEditor

Creating the authorization policies

Adding some data in order to make decisions

Deploying a Docker-based OPA server

Testing the authorizations

Challenges with OPA

Summary

Part 3: Building a Blueprint Application with .NET

14

Decomposing the Functional Responsibilities

Some more context about the DemoEditor information system

Current company organization

The current state of important data

What kind of information system is going to be built?

Decomposing into a four-layer architecture

Business process management layer

The BCM layer

Software layer

Hardware layer

Looking for standards for interfaces

Writing up the specifications for the different modules

Overall system specifications

The books referential service

The issue with migration

Authors referential service

IAM

Authorizations

Technical specifications

Explaining how the interface will run the main use scenarios

Integration of the applications in the system

Technical implementation of the applications

Summary

15

Plugging Standard External Modules

Externalizing IAM with Keycloak

Realms

Delegation to Office 365 for internal user SSO

Minimal implementation of external users

Externalizing persistence with MongoDB

Choosing between local and online storage

A little digression about old CPUs

No schema, no initialization phase!

Securing through regular backups

Optionally installing a web console

Externalizing document management

Choosing the right standard, not the right implementation

Starting up Alfresco

Alternative for Alfresco setup

Testing calls to the EDM module

Using the Apache Chemistry console as an Alfresco CMIS client

Externalizing queue management with RabbitMQ

Starting up the MOM

Defining which streams should be handled, and operating a MOM

Summary

16

Creating a Write-Only Data Referential Service

Technical warmup

Getting connected to the Git repository

Creating the project skeleton

How to run the example

Starting small, with a RAM-only application

Representing the data

Reading a single piece of data

Adding persistence

Storing operations rather than states

Choosing NoSQL

A few more words on NoSQL and persistence

Remark on multitenancy

Creating the data business model

A switch in thinking – modify first, create afterward

Linking the books-changes collection to the database

Finishing the patch operation implementation

Creation as a special case of modification

Summary

17

Adding Query to the Data Referential Service

Adding features to the first version of the data referential service

Testing the data referential service

The particular case of deletion

Querying as an extension of the reading responsibility

Using ODP for a standard query API

Pagination, filtering, and more

Links to business intelligence and big data systems

A few last remarks on the books API implementation

Testing

Testing at the API level

Managing dependencies

Summary

18

Deploying Data Referential Services

Installing the data referential service in the information system

Docker deployment

Adding a status operation to the API

Plugging the data referential service into the IAM

Preparing the IAM to be used by the books referential service

Integrating the code with the IAM

Adding RBAC authorization based on IAM attributes

Polishing with additional ABAC authorizations

Why are we doing all this?

Summary

19

Designing a Second Data Referential Service

Building the authors data referential service

Links between data referential services

Adding a local cache

Using Webhooks for cache quick refresh

Implementing callbacks in the authors referential

Registering to webhook in the books referential

Using the cache in the books referential

Additional issues on webhooks management

Dealing with eventual consistency

A data referential service is not just about CRUD

Summary

20

Creating a Graphical User Interface

A standard web single page application

Defining the GUI

Preparing IAM for the client

Generating the client skeleton

Taking into account authorization concerns

Adding a debugging identity page to the application

Connecting the GUI to the backend

Modifying authorization in the backend

Listing books from the portal GUI

Solving authentication issues

Separation of concerns for the pagination feature

Implementing pagination

Summary

21

Extending the Interfaces

Importing some real data

The first step of migration

Implementing data import

Testing data import and next steps

Testing the GUI

Automatic end-to-end tests of the GUI

Tests at the component level

Applying a MAUI mobile application

The benefits of a responsive interface

MAUI Blazor as a mobile platform

Fusing MAUI and Blazor WebAssembly

Additional considerations for graphical interfaces

Decomposition of the frontend

Content negotiation for a simple interface

State management

Separation of responsibilities for application logging

Summary

22

Integrating Business Processes

The need for processes

Orchestration and choreography

Application to DemoEditor

How to choose the right process implementation

The GUI as the main process implementation

Human processes versus automated processes

Implementation through the user interface

Passing the message to another role in the process

Letting the author fill in the information in the same GUI

Validating and publishing information

Advanced uses of the GUI for process implementation

Implementation through a wizard

Dealing with long-duration processes

Externalizing processes in a dedicated tool

Using a BPMN approach, but not a BPMN engine

A light alternative to BPMN orchestration

Choreographing the regular import of the book’s catalog

Orchestrating the creation of a new project

Adding a dedicated service for a complex step in the processes

Business-aligned definition of a middle office

Defining the middle office service API

Implementing the middle office service

Plugging the middle office into the process of book contracting

Summary

Part 4: Validating the Architecture

23

Applying Modifications to the System

Technical requirements

A word of warning, and an anecdote

Modification at the data level

Adding a simple attribute

Extending the possible values of an existing attribute

Migrating a schema in a schemaless world

Ascending compatibility and OpenAPI

Modification in the GUI

Taking into account the addition of a new attribute

Modifying ergonomics of the GUI

Adding a change tracker in the form and implementing the auto-patch

Adjusting the behavior of the page

Considering further improvements

Adjusting IAM and business rules

Common IAM changes

Industrializing Keycloak

Modifying the identity sources

The challenges of time and compatibility in IAM adjustment

Modifying the business processes

Types of modifications

Examples of business process modifications

A few last words on the sample software applications

Code adjustments

Adjusting exposition

Summary

Index

Other Books You May Enjoy

Preface

Thanks for acquiring this book! It was born out of the need to industrialize Information Systems. Software is everywhere but, apart from a few high-cost applications, it is full of bugs and costs a lot in maintenance, and most developers spend half their time reinventing the wheel or fighting against the Information System to add some new features.

This is my harsh observation from more than 25 years in the field and a few years working specifically on digital transformation for many different organizations. These diverse experiences convinced me that the root problem is always the same: the software industry does not actually follow the rules of industrialization, which are to decompose systems into smaller ones and join modules together with standards-obeying interfaces.

Working together with academics helped me formalize these problems of business/IT alignment, along with their sources and possible solutions. But most of my customers have told me that I stood out as a consultant because I was able to not only provide boards of directors with a clear transformation plan but also to accompany it with advice on semantics all the way down to the API contracts, and even some hands-on help in the code for the hard bits. They said this made the whole partnership more tangible and provided visible business results, so this is how I wanted this book to be: some theory first because we need to switch the way we think and create Information Systems, but some code as well because too many of them are not efficient today.

Having seen the positive effects on the business efficiency of many companies, from a small lawyers firm to industry/logistics enterprises and to some huge farming cooperatives, I am convinced applying industry concepts to software design and architecture will become a more and more cost-effective approach, particularly since software and data are becoming the competitive backbone of most organizations nowadays.

Using norms and standards for API interfaces, externalizing functions such as authentication, authorization, electronic document management, and, most importantly, thinking in terms of business functions rather than technically, is of utmost importance if you want to transform an Information System from a cost center to a strong asset of an organization.

This method has helped many companies and government-owned agencies, and I hope it will do the same for your Information Systems!

Who this book is for

This book aims to help you create or evolve Information Systems architecture in such a way that it becomes more flexible and provides more business value to your organization. Since it is theoretical, business-oriented, and technically applied with an application in .NET, it can be read at many levels and by many profiles:

Board directors and chief officers can read the first chapters in order to understand digital transformation and how it can be strategized. Finding out about the most common problems in Information Systems should help them invest in the right places and prioritize work, taking into account not only features but also technical debt reduction when justified.Tech leads, architects, and R&D managers will also benefit from reading the next chapters that make the strategy more tangible and explain best practices in structuring Information Systems, choosing tools and methods, and applying alignment in their job.- Finally, developers or hands-on architects and technical directors will continue reading the book's last chapters that really put the previous chapters into practice by coding the different concepts, plugging external services, and orchestrating API and webhooks together to create a small but working sample system.

What this book covers

Chapter 1, The Sad State of Information Systems, explains what prevents most Information Systems from providing as much value as they should. It also proposes some methods to measure the efficiency of an Information System, focusing on what causes are behind most of the problems observed.

Chapter 2, Applying Industrial Principles to Software, starts by explaining what characterizes the principle of industrialization, and then proposes a way to apply a transformation to Information System management. It focuses on the importance of standards and norms to reduce complexity, which is quite high in most virtual systems.

Chapter 3, Reaching Business Alignment, brings a layer of theory to the problems analyzed in the previous chapters, explaining Conway’s law and some other principles, patterns, and anti-patterns in business/IT alignment. This chapter also introduces the four-layer diagram, which will be used throughout the rest of the book.

Chapter 4, Dealing with Time and Technical Debt, describes the concept of technical debt and explains how Information Systems often start quite well adapted to their context but slowly drift apart from efficiency over time. It also reflects on how to adopt an Agile approach to architecture.

Chapter 5, A Utopic Perfect IT System, is about setting a vision of an absolutely perfect Information System that would be completely aligned with business needs and where changes would be as easy as the original design. Even if this utopic system will never exist in reality, it will help you understand how externalizing functions and strictly separating responsibilities can make existing Information Systems more efficient and ease their evolution over time.

Chapter 6, SOLID Principles from Code to Systems, starts a more technical set of chapters. It discusses the SOLID principles, which usually apply to code, and adapts them to systems architecture. We also present the example Information System that will serve as an applied example for the rest of the book.

Chapter 7, C4 and Other Approaches, lists some of the most well-known architectural methods in software design and architecture. It analyzes the differences and common points between them, giving you some ways to choose which is best suited depending on the context.

Chapter 8, Service Orientation and API, starts with a historical description of all the techniques that have been used to implement the concept of a service, explaining what people expect from the concept. It will explain how standards and pivotal formats are useful in services.

Chapter 9, Exploring Domain-Driven Design and Semantics, is about the importance of having a clear definition of the semantics of a business domain. Though this may sound a bit distant from the notion of software architecture, it actually is at the heart of the method of business/IT alignment.

Chapter 10, Master Data Management, provides a structured view of the management of data in a well-aligned Information System. Master Data Management is much more than persisting data in a database. It is about defining its life cycle and the associated governance and ensuring a progressive evolution of data management, particularly when some legacy data repositories are already in place.

Chapter 11, Business Processes and Low Code, explains the place of business processes in an Information System, and how they can be described formally using standards such as BPMN. Though this is still a theoretical chapter, it describes the tools that can be used in order to implement business process management in an Information System.

Chapter 12, Externalization of Business Rules, closes the loop on the three parts of the ideal IT system, after data and process management. It explains what a business rules management system is and shows some examples of implementations.

Chapter 13, Externalization of Authorization, explains the concept of identification and authorization management and shows how it can be externalized to the benefit of the evolution of the whole Information System. Paradigms such as Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC) are also explained with examples.

Chapter 14, Decomposing the Functional Responsibilities, is where we start applying the knowledge from the previous chapters to an example; we’ll get our hands dirty with the code and services. The chapter starts by using the four-layer diagram to model the system and then prepares the technical specifications for the features that will be better considered in external applications.

Chapter 15, Plugging Standard External Modules, takes these specifications into account and proposes some off-the-shelf software solutions for the different features that have been identified, such as Apache Keycloak for IAM, MongoDB for persistence, Alfresco for electronic document management, and so on.

Chapter 16, Creating a Write-Only Data Referential, focuses on the business domain data management of the example Information System. It shows how to code an API server in .NET that will serve the data needed in a way that as far as possible respects standards and all the principles of business/IT alignment that have been described previously.

Chapter 17, Adding Query to the Data Referential Service, builds on the previous chapter and adds some features to the data management system that has been put in place, but also shows how it can be tested in an industrial manner. Standards such as the Open Data Protocol are used in this chapter to improve the implementation.

Chapter 18, Deploying Data Referential Services, shows how to deploy the services that were created in the previous chapters and how to plug them into the IAM. Since security is such an important and complex subject, this is the subject of this dedicated chapter.

Chapter 19, Designing a Second Data Referential Service, uses all the principles presented in the previous chapter to build a second API implementation, this time complexified by the fact that this service has to use webhooks and create a local cache of data in order to reduce calls to the first data referential service. Though it does not add any new principles, it does show how loose coupling can be realized in practice.

Chapter 20, Creating a Graphical User Interface, adds some graphical interfaces to our example Information System, which only exposed API endpoints for now. After showcasing a web-based single-page application and explaining how the concepts and recommendations of business/IT alignment also apply in the GUI domain, it details connecting the data pagination mechanism with the backend, observing a rigorous separation of responsibilities.

Chapter 21, Extending the Interfaces, builds on the previous chapter and applies some industry standards to the GUI that was previously built, namely by automating tests on it but also by showing how importing data can be automated to migrate a system from its legacy data management. It finishes by showing how a good separation of components can help, with the right technical stack, to quickly build a mobile application.

Chapter 22, Integrating Business Processes, closes the set of chapters corresponding to the example application by explaining how business processes can be integrated into it through the GUI, a BPMN engine, or a low-code/no-code modern orchestration tool such as n8n. It finishes by explaining when a dedicated service is necessary to implement orchestration or choreography.

Chapter 23, Applying Modifications to the System, sums up the whole idea of the book by showing how the Information System that has been built can now support all kinds of changes by being impacted by side effects or losing its ability to evolve further. In this chapter, we will change the data structure, the GUI, and the business rules, particularly authorization, and even adjust the business processes, hopefully demonstrating that the initial planning of the architecture makes all this easy.

To get the most out of this book

Docker has been used in order to reduce the required tooling as much as possible. Since all images are available online (even the custom ones created for the example information system, available at https://hub.docker.com/repositories/demoeditor), all you need to work with the application is Docker (see https://docs.docker.com/engine/install/ for the installation instructions). The image versions follow the branches of the code.

If you want to debug the application and make some changes to it in order to follow the instructions from the book, then you will also need .NET 8.0 SDK (https://dotnet.microsoft.com/download), Visual Studio Code (https://code.visualstudio.com/download), and Git (https://git-scm.com/book/en/v2/Getting-Started-Installing-Git). Postman (https://www.postman.com/) will also be used to quickly inject data.

Software/hardware covered in the book

Operating system requirements

.NET 8.0 SDK

Windows, macOS, or Linux

Visual Studio Code

Git

Docker

Postman

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Enterprise-Architecture-with-.NET. If there’s an update to the code, it will be updated in the GitHub repository. This GitHub repository is made with branches that correspond to steps in the building of the examples in this book: https://github.com/PacktPublishing/Enterprise-Architecture-with-.NET/tree/main/DemoEditor#versioning.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Imagine we use a link between a book and an author entity.”

A block of code is set as follows:

{     "rel": "author",     "href": "https://demoeditor.com/authors/202312-007",     "title": "JP Gouigoux",     "authorMainContactPhone": "[phone]" }

Any command-line input or output is written as follows:

catch (Exception ex) {   transac.Rollback();   throw new ApplicationException("Transaction was cancelled",ex); }

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Once connected, you will be presented with the welcome page interface, which you can return to at any time by clicking on Business Central, or the home icon in the top-left part of the screen.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com

Share your thoughts

Once you’ve read Enterprise Architecture with .NET, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781835085660

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Business-Aligned Architecture and the Problems It Solves

This first part of the book is not technical at all and aims to provide a good understanding of what business-/IT-alignment is all about. It explains the difficulties and shortcomings in existing information systems and how some industrialization approaches may help overcome these problems. This is useful for anyone confronted with digital transformation. People with an IT system that does not deliver on its functional promises is hard to evolve, and is not cost-effective should also get a better understanding of why this is the case. After explaining causes and symptoms, this part of the book proposes a theoretical approach in the form of a utopic system that would be free of all these shortcomings and will guide our work in the next parts.

This part includes the following chapters:

Chapter 1, The Sad State of Information SystemsChapter 2, Applying Industrial Principles to SoftwareChapter 3, Reaching Business AlignmentChapter 4, Dealing with Time and Technical DebtChapter 5, A Utopic Perfect IT System

1

The Sad State of Information Systems

Before jumping to solutions, it is essential to share a thorough diagnostic of a situation. In the case of information systems and, more generally, computer use, any user knows of the term “bug” and has experienced the frustration associated with malfunction, sometimes with a high personal impact (loss of personal data, consequences regarding revenue, and so on). For companies, a malfunction in IT can have harsh consequences since they depend more and more on computers to realize their business operations, hence their financial objectives.

After defining what an information system is and explaining how its efficiency (or lack thereof) can be calculated, an attempt at classifying the causes of such problems will be exposed. As for solutions, this will be the subject of the rest of this book. But for now, we must understand what is going wrong with information systems, how this happens, and – more importantly – why.

In this chapter, we will cover the following main topics:

What is an information system?Why is software building still a craftsmanship, with good and bad consequences?How the efficiency of an information system can be evaluatedHow to classify the different impacts that can happen on information systems, as well as what their causes and consequences are

What is an information system?

Before talking about the state of information systems, it might be useful to give a clear definition of what an information system is and even what a system is.

A system is a group of items that operate together to reach a common goal. This is the basic difference between a system and a union of individualities: the parts of a system work together toward a vision.

An information system (we will sometimes abbreviate it as IS) furthers this definition as a group of items that share information to reach a common objective. Strictly speaking, an IS is not necessarily made of software, even though most of what we will talk about in this book is about computerized information systems. And, even in the most sophisticated IS, there remains a non-neglectable part of information that is not software-contained. This is a situation that we will talk about, but for the main part of this book, we will put ourselves in the hypothesis of software-based information systems as they are now pervasive in almost every company and organization.

So, an IS is understood as a set of software tools that operate toward a goal. This goal is typically designed as a business process for most of the companies that own the system. It should be noted that software always depends on hardware, but this tends to be more and more hidden in the background and the IS is more and more considered as the software means, organized together to reach a business goal, by implementing functional processes efficiently.

A quick history of information systems

When dealing with general subjects, such as the quality of information systems, it is always interesting to have a look at the past and analyze the evolution toward the current situation.

If we follow the definition stated previously, information systems only appear when at least two entities collaborate. In our hypothesis of software-based information systems, this means at least two computers have been connected. This means that initial room-sized computers that are operated on-premises should not be considered systems, though they were often called “large systems” (the items that were assembled, in this case, were computation mechanisms, short-life memory, and long-life memory).

Let’s go forward in time a little bit and talk about IBM’s client-server mainframes: those are the first ones that we can consider as information systems since there were client stations connected to a central computer, with information flowing between them. The protocol was proprietary, which made it easier for IBM to reach the high quality of the service provided by these systems (lots of them are still in operation nowadays). Since a unique implementation of a protocol was defined by the same team, compatibility, and interoperability were a very light issue. This is legacy, but when the system is working and when modernizing it is highly risky, the businesses logically make the choice of not moving anything (this is the first rule in information systems management: if it works, do not touch it).

Fast-forward again and we are in the nineties. This is the era of personal computers (PCs). Though there is a worldwide attempt at keeping compatibility between the machines, anybody using computers at this time who was unable to use a given piece of software, because it did not support the embedded video card, knows this was partly a failure. Of course, things have greatly improved since then, with Video Electronics Standards Association (VESA) international standards being created for video cards and screens, such as VGA or VESA, and lots of other norms that make it possible, in the present time, to change components of a PC without breaking down the whole system. Still, the toll on information systems has been quite high: how can we expect networks of machines to work well together when even a single machine’s components are hard to assemble? Proprietary formats, different encodings, an almost complete absence of strong data exchange protocols: everything led to a difficult situation, and only high-budget companies could operate complex computerized systems with the help of experts, who would twitch about anything from jumpers on an electronic board to compilers parameters to make the system work.

Fortunately, Y2K came with the internet expansion and its radical approach to normalizing the exchanges between computers. Now, every single computer that abided by TCP/IP, HTTP, Unicode, and other internet-born standards could exchange data with another one anywhere in the world, regardless of its hardware and operating system implementation. This is one of the biggest steps forward in the history of IT and the root definition of what our “modern” information systems are today.

A few software layers were added to this to make it easy to reuse functions in a system. It started low, with local reuse of code through routines, then libraries and components. With the advent of network capacities, this evolved to distributed components, web services, and finally service-oriented architecture (SOA). This is also the time when n-tier architectures were put in place, establishing a first layer of responsibility separation inside the software application, between graphical user interface (GUI) management, the exposition of functional services, the implementation of business rules, and persistence management.

The last bit, SOA, has led many companies to costly failures, and the vast majority of attempts at putting such architectures in place resulted in important financial loss and abandonment of projects. The technological step was too high to operate smoothly, and lighter alternatives have appeared to remove local difficulties of the SOA approach:

Standardized message-oriented middleware: To counter the proprietary exchange protocols put forward by large software companies operating on SOA and using it to lock their customers in their systemThe REST approach: To lighten the weight of SOAP/WSDL web services and the weight of all associated normsEnterprise Service Bus: This technique is used to reduce the importance of the middleware and reach a “dumb pipes” paradigm where the software applications participating in the system would be able to communicate with each other without needing a central piece of software that would necessarily become a single point of failure

As the presence of a few reference books (Sassoon, Longépé, and Caseau) show, the best practices to design a strong and evolution-capable IS were already available in the late nineties, though not very well known. But it was during the 2000s that these practices came to a larger share in the community and SOA and other service-based approaches flourished, leading to the microservices architecture in the beginning of the 2010s. This set of practices is still considered as the reference architecture at the time of writing, though we will see that not all its recommendations should be applied without a strong analysis of its usefulness in the studied context. As we will see, the granularity of services is key to obtaining an efficient IS. But for now, we will talk about software building in general and try to understand the current limits of this so-called “industry.”

Software building – still craftsmanship

The precise definition of an information system, as interpreted in this book, has been given together with a brief history of its evolution. This history is not only very short but shows many recent evolutions, most of them radically different from the previous state of the art. This very rapid evolution is a sign that information systems design is not something that can be considered stabilized and completely understood.

There remains a large part of craftsmanship in the design and deployment of a software information system. Craftsmanship has its advantages: human attention to detail, custom-tailored functions, uniqueness, and more. It also has numerous drawbacks, such as high costs, difficulty in evolving in a controlled way, dependence on a few creators, and many others. These drawbacks outgrow the advantages in modern companies for which the information system has become the operational backbone.

Well-crafted information systems are an evolution from arbitrarily evolving ones, and there is nothing to be ashamed of with a craftsman’s job, but the way forward today is toward an industrialized approach to information systems. This is what this book is all about.

Craftsmanship as opposed to previous lack of quality

Craftsmanship is used in several domains of IT where opposition is established with older, more arbitrary, and self-organized methods where used. For example, numerous IT conferences include the word “craftsmanship” in their names as a statement of their will to address quality and heterogeneity issues.

Information systems have long appeared simply because pieces of the system were put together and linked without any reflection on the whole system itself. This is generally what happens with so-called “point-to-point integrations,” where connections are made between different software modules while considering only the source and the destination of the link, without any map of all the links that exist, sometimes reproducing an already existing link or reversing the initially intended direction of the functional dependency between the two modules.

Such systems that are born without anybody thinking of the whole functioning have very few chances to remain stable for long. In the rare cases where only a few connections are created, the system can operate fine, but we all know that IT evolves very quickly, and business needs are added all the time (“the only stable thing in a system is its need to evolve”). If nobody has a global view of the whole system, there is no way that its overall evolution is going to develop ideally. It would be pure chance and Murphy’s Law states that, if something can go wrong in a software system, it most definitely will.

Software craftsmanship involves a will to not let the system create itself and develop in the wrong direction, but to take extra care of the software quality and methods to build time-enduring and evolution-capable systems. When applied to code (where it mostly happens), software craftsmanship includes test automation, a refactoring approach, quality indicators monitoring, and many other methods and techniques that go beyond minimal practices.

We could argue that these practices are contrary to craftsmanship, but this is only true in established industries, where craftsmanship lives in opposition to standardized industrial production. In the world of IT, industrialization has not happened yet. The word “craftsmanship” has been used by artisans of IT who pride themselves on analyzing problems and solving them with code more elegantly and sustainably than usual slack in their analysis and in the realization (many of us have seen code without any analysis documentation and a single unit test, and yet that has been sent into production just because “it worked”). Craftsmanship – as the word is used in the IT movement – could even be said to be the very first step into industrializing the software domain since this is the first resolute action into making code clean and not letting it derive into what is commonly named in the IT jargon as a “big ball of mud.”

It might look like I oppose craftsmanship and industrialization, but one isn’t better than the other: they are simply two phases of the development of a domain – in our case, software. Personally, after 38 years of programming with the desire to do things cleanly, I consider myself a craftsman; my goal with this book – and the past 15 years of my architect career – is to humbly help make another step so that software gets out of its infancy problems and becomes an adult industry.

As a side note, a subject of discussion among software architects is about our working domain to be able to ever become an industry. I tend to think that the amount of creativity needed to be a good software engineer will make it impossible to fully industrialize software production, but that a lot of things should be industrialized to reach this mature, adult phase, where IT will finally deliver its complete value.

A word about “emerging architecture”

The concept of emerging architecture is that, in some human constructions (mainly software applications), architecture is not established upfront, but appears gradually with the construction of the object. This concept is often used together with Agile methods, where the software construction is iterative and not done in a single succession of phases, such as in the “V-cycle” method. Instead of a single design/development/test series of steps, Agile software development loops many times over these steps, each time building on the previous cycle to gradually work toward a final vision that may even evolve during the steps.

In this case, each step involves the minimal design activity necessary to realize the cycle, in a Keep It Simple mindset. Thus, there is no initial complete vision of the final architecture, which sometimes can be seen as a serious limitation of Agile methods, but is at the same time their force, as they continually adapt. There are ways for this to go awry, though, and it mostly happens where participants in the project expect the architecture to emerge naturally. This confusion is often seen in software developments where no architect is involved and where developers believe individual best practices, goodwill, and craftsmanship will have a quality impact on the outcome. The reality is that these practices will positively affect the individual results of each Agile step but will not guide the overall architecture in any sound place since there is no long-term direction.

Therefore, it is very important to understand that emerging architecture does exist but necessitates an active engagement in making it happen gradually. It naturally happens at the module level, where a single developer carefully refines and refactors the code. But to work at the system level, there needs to be the same level of engagement.

Craftsmanship as opposed to an industrialized approach

In the previous section, we have opposed craftsmanship to lack of quality intention in the code and thus shown how positive craftsmanship can be. Here, we will point out its limits when opposed to industrialization. Craftsmanship bears the idea of a highly skilled individual operating carefully, where time spent is of little importance compared to quality.

Though craftsmanship – in its noble sense of dedication to hand-polished, high-quality work – is worth praise, it also states that the level of maturity of the domain is still low. When reaching maturity, disciplines tend to separate different steps of the work, automate some of them, standardize practices and tools, and overall become more efficient, even bringing quality to a higher level that cannot be obtained with an individual human approach alone. Moreover, industrialization normalizes the whole process and makes it possible for anyone following the norms – and not only highly skilled people – to reach this high level of quality.

This is what has been done in all industries (and this is even why we call them this), and it is the natural evolution human workers try and reach. In the software field, predictability, quality, and time-to-market are sought-after qualities and an industrial approach is necessary to reach them.

It is essential to point out that there is nothing wrong with a non-industrial approach. Software is not an industry yet. After all, bridges have been built for more than 4,000 years, so it is quite understandable that this has become something of a controlled and well-established way of working. Software building, on the other hand, has only been there for a few decades and still is in its infancy.

But the important information here is that craftsmanship, despite all its advantages, is the step before industrialization, and lots of information systems owners nowadays really crave an IT team that reaches this next step. Stakes are huge for some of them, where competitive advantages come mostly from the information system. It has been said that “all companies today are software companies” and this stresses once more the importance of the information system and the absolute necessity of reaching higher levels of quality.

Concept of technical debt

Technical debt is a concept that has been created to explain, through a metaphor, how the low quality of software development can negatively affect its future development. In financial debt, you must pay regular interest, depending on the amount you have borrowed. In the software side of the metaphor, buying some time by cutting corners and lowering overall quality will have to be paid regularly, so long as the low-quality module remains active. Bug correction and maintaining the module will take some time, and the team will not be able to spend on new features that have value for the users. The lower the quality of the module, the higher the “rate of interest” – in our case, the time spent on maintenance. In the worst case, the quality of the software is so low that all available money/development time is spent only on paying for the interest of the debt (keeping the application running), which means that no money remains available to reimburse the debt/correct the software, let alone paying for features with more value.

This concept will be discussed at length in Chapter 4, but since we are talking about craftsmanship, now’s a good time to explain the link between both concepts right away.

Craftsmanship is often seen as a software development approach where technical debt is maintained at its lowest level possible, sometimes being virtually nonexistent. A good craftsman developer will take pride in having zero-defect software, 100% coverage via automated tests, with a fully automated integration and deployment system.

The industrialized approach that was opposed to craftsmanship results mostly in an improvement of the overall quality, but also goes beyond in how it deals with technical debt. Where craftsmanship tends to have an individual and binary approach toward technical debt, as opposed to careless development that lets it potentially go unleashed and out of control, the industrialized approach manages technical debt. The financial metaphor still stands: instead of refusing any debt altogether, a well-thought operator will carefully manage their capital, borrowing if the advantages are indeed higher than the costs of the loan. Compared to a craftsman, an industrially-oriented developer will be more aware of the importance of time-to-market and assume a controlled level of technical debt if it helps them reach users before their competitors, thus bringing benefits that will partly be affected in reducing the technical debt afterward.

The long-standing comparison of software to mechanical systems

Comparison of software and mechanical industries (in most known cases, the automotive industry in particular) is such a habitual position of contenders of both sides that it has been worn out. Also, comparisons do not necessarily bear logic, depending on the way they are realized. For example, in the famous Microsoft versus General Motors meme, comparing a single car with a single piece of software brings strange conclusions from each side (If the car industry evolved as quickly as software, we would have cars driving 1,000 miles per gallon”/“If car industry worked as the software industry, a car would crash inadvertently every 1,000 miles”). The underlying error in this ill comparison is that the level of complexity is not the same on both sides. If you were to compare a single car created in an industrial factory, the cost should be compared to the single operation of a functional process in an information system, since only one function is individually operated (transporting a few people in the case of a car, calculating a payroll – for example – in the case of the software application). If we turn the problem the other way around and want to compare the internal operation of the software with its tens, maybe hundreds of thousands of lines of code, the right comparison would be with the car factory itself as it has the necessary modularity to change its car model production, and it also contains thousands of functions and many more moving parts to perform such a moving task.

In short, wrongly-calibrated comparisons can be very misleading and of little practical interest. But comparing information systems building to the design of a car factory, for example, is closer to the reality of the complexity of each system and can provide interesting insights, so long as you contextualize them carefully. As stated previously, comparing information systems to bridges can be pertinent if we stay on the criteria of the age of the industry. How could IT, an activity that is only a few decades old, reach the same level of maturity as an activity that has been developed over several millennia?

We will stop this comparison for now, but you will be introduced to several comparisons between the software “industry” and more traditional industries that better deserve this qualification throughout this book. We will try to keep the comparison as helpful and legitimate as possible. Again, there is no judgment in saying that IT is not completely industrialized yet: some systems undoubtedly are, and some are not. And this should not be taken against anyone as industries took many human generations to reach their current level. Software-based information systems simply have not had the time to do so. This book is about the means to help go in this direction, taking into account experiments from other industries, while keeping in mind that comparisons may sometimes be misleading and should be used while paying attention to their applicability.

Now that we have established what an information system is and the many infancy problems that it may have because the field is not industrialized yet, a sound engineering approach is needed to evaluate this lack of maturity.

The efficiency of an IS

Engineering is about making things in a controlled way; information systems will not escape this much-awaited transition as their efficiency could be improved. To do so, we need indicators, measures, and a method to get the results on a given information system. Fortunately, there is a consensus on this in the field of expertise, as we will see now.

The measure of efficiency of a system

Since a system is a group of items working together toward a goal, as defined previously, measuring the efficiency of the whole means more than just knowing the efficiency measures of each item and summing them one way or another. It is often said that the value of a good information system is much more than the value of each of its moving parts and that the most positive effects come from their interactions.

A good way to measure the efficiency of an information system is to evaluate how much time and money it helps save on the functional processes it supports. However, this is quite complicated to measure since a single IS generally operates several processes, and their respective importance and costs should be evaluated. If you consider that processes are also partly human-operated and that the gain of efficiency does not come only from the software part but from the way the functional teams use the application, how they have been trained, what investment has been put in the hardware for performance, and so on, it starts getting difficult to evaluate the efficiency of a complete system with these metrics. Furthermore, the output may be closer to a return-on-investment calculation than that of an efficiency metric.

This is why the efficiency of IS is often evaluated through simpler, more attainable metrics, namely the percentage of cost used for maintenance. This simple ratio allows us to know how much money is put to keep the system working (this is an operational cost) versus how much has been put into designing it and making it better (which is an investment cost). As functional features are what is asked of the system, the more maintenance there is to keep it operating, the less money is related to getting value out of the system. Of course, there are many points of view on software efficiency, and we are going to browse a lot of them together in this book. But maintenance is simply the easiest way to start evaluating the state of an IS.

The cost of maintenance

Maintenance