The Standard defines the software engineering process in three main categories: Purposing, Modeling, and Simulation. Each aspect is crucial in guiding engineering efforts toward a successful solution and fulfilling a purpose.
The order in which these aspects are followed is also intentional. A purpose must exist to shape the modeling process, and one can't simulate interactions without models. But while that order at the initiation of the engineering process is crucial, it's important to understand that the process is selectively iterative. A change in the purpose may reflect a change in the simulation. Still, not necessarily the modeling, and a change in the models may not necessarily require changing the purpose or the simulation.
The purposing process is our ability to find out why we need a solution. For instance, if we have an issue with knowing how many items are on the shelf in a grocery store, we deem the manual counting process inefficient, and a system needs to be implemented to ensure we have the proper count of items.
Reasoning relies heavily on our ability to observe problems and then articulate a problem to devise a solution that addresses the given problem. Purposing, therefore, is to find a reason to take action.
So, we have the observation, the articulation of the reasoning (the problem), and the intent for a solution. All of these aspects constitute the Purposing portion of engineering software.
We live in a world full of observables. Our inspiration is triggered by our ambition to achieve more. Our dreams reveal blockers in our way that we need to solve to continue our journey and fulfill our dreams. From the moment a young student uses a calculator to solve a complex equation to the moment that same student becomes an astronaut, calculating the trajectory of satellites orbiting our planet.
Observation is our ability to detect an issue that's blocking a goal from being achieved. Issues can be as simple as having the proper count of items on a grocery store shelf or as complex as understanding why we can't capture images of planets millions of light-years away. Engineers would describe these as observable problems.
The greater the purpose, the more complex a problem will be. But starting with more minor purposes is a way to train our minds to tackle bigger ones—step by step, one problem at a time.
Describing the observable is an art in and of itself because describing a problem well is halfway to its solution. The clearer the articulation of the problem, the more likely it is to be understood by others, helping us to solve that very same problem.
Articulation is only sometimes with words. It's also with figures and shapes. It is not an accident that some of the most advanced ancient cultures have used figures and shapes to describe their times and history. Figures are a universal language, understood and interpreted by anyone who can relate to them much faster than learning a spoken language. A figure or shape might be the most optimum way to illustrate an idea, as its pictures are worth thousands of words.
Articulation requires a passion for solving the issue, whether written, spoken, or illustrated. A passionate mind conveys the hidden message of the criticality of the problem to be solved. Articulating a problem is a big part of selling a solution. Our ability to convey an idea to other engineers and those investing in and using this solution is one of the most critical aspects of engineering software.
A part of the purpose is the way to fulfill it. In the engineering industry, fulfilling the goals can be done by more than just any- means. Software fails worldwide because the solutions aspect was overlooked as a trivial part of the purpose. You may have heard of engineers up against a deadline who decide to cut corners to achieve a goal. In our Standard, this is a violation. A solution must not simply reach a goal but must be a purpose in and of itself to aid ambient architectural issues such as optimization, readability, configurability, and longevity. Solutioning is part of the purpose of software craftsmanship.
Modeling is the second most crucial aspect of software engineering. We can extract models from the actors in any problem, whether these actors are living beings, objects, or others. We extract only the attributes relevant to the problem we are trying to solve and discard everything else. For instance, when trying to count the items on a grocery store shelf, we would need a model for these items.
A more straightforward example would be detecting perishable items in a grocery store. The only attribute we are concerned with here is the expiration date on the item. Everything else, including the label, color, weight, or any other details, is outside the scope of the modeling process and the solution.
Modeling, then, can only exist with a purpose. The purpose defines the scope or the framework of which the modeling should occur. Modeling without a purpose leaves the door open for attracting an infinite number of attributes every single element in the observable universe may have.
The relationship between the purposing and modeling attributes is proportional. The more complex the purpose is, the more likely the modeling process will require more attributes from the real world to model in the solution.
We express our models in programming languages as a class. The aforementioned perishable items problem above can be represented as follows:
public class Item
{
public DateTimeOffset ExpirationDate {get; set;}
}The name of the class represents the overall type of the item. Since all items have the same attribute of ExpirationDate, the name shall stay as generic as possible.
Now, imagine if our purpose grew more complex. Let's assume the new problem is identifying the more expensive perishable items so the store can put them up front for selling before the less costly items. In this case, our model would require a new attribute such as Price so a computer program or a solution can determine which is more valuable. This is what our new model would look like:
public class Item
{
public double Price {get; set;}
public DateTimeOffset ExpirationDate {get; set;}
}Models govern the entire process of simulating a problem (and its solution). Models break into three main categories: Data Carriers, Operational, and Configurations. Let's discuss those types in the following sections:
Data carrier models have one primary purpose: to carry data points across systems. They can vary based on the type of data they carry. Some carry other models to represent a complex system, while others represent references to the original data points they represent.
Data carrier models in a relational fashion can be broken into three different categories. These categories make the priority development, design, and engineering areas much clearer. For instance, we can only start developing secondary/supporting models if our primary models are in place first. Let's talk about these categories in detail:
Primary models are the pillars of every system. Any given system can only proceed in design and engineering with a clear definition and materializing of these primary models. For instance, if we are building a schooling system, models like Student, Teacher, and Course are considered Primary models.
Primary relational storage schema models do not contain foreign keys or references to any other physical model. We call these models Primary because they are self-sufficient. They don't rely physically on some other model to exist. This means that a given primary model like Student may still exist in a schooling system, regardless of whether a Teacher record exists or not. This is called physical dependency.
Primary models, however, may rely conceptually or logically on other models. For instance, a Student model has a logical relationship to a Teacher, simply because there can never be a student without a teacher and vice versa. A Student model also has a conceptual relationship with its host and neighboring hosting services. For instance, there's a conceptual relationship between a Student model and a Notification model regarding business flow. Any student in any school conceptually relies on notifications to attend classes and complete assignments or other events.
On the other hand, Secondary models have a hard dependency on Primary models. In a relational database model, secondary models usually have foreign keys referencing another model in the overall database schema. But even in non-relational storage systems, secondary models can be represented as nested entities within a given larger entity or have a loose reference to another entity.
Let's talk about some examples of secondary models. A Comment model in a social media platform cannot exist without a Post model. You cannot comment on something that doesn't exist. In a relational database, the comments model would look something like this:
In the example above, a secondary model Comment has a foreign key PostId referencing the primary key Id in a Post model. In a non-relational system, secondary models can easily be identified as nested objects within a given entity. Here's an example:
{
"id": "some-id",
"content": "some post",
"comments": [
{
"id": "comment-id",
"content": "some comment"
}
]
}Secondary models may generally have logical and conceptual relations to other models within their host, neighboring, or external systems. However, their chances of having these conceptual relations are much less than those of Primary models.
Relational models are connectors between two Primary models. Their main responsibility is to materialize a many-to-many relationship between two entities. For instance, a Student may have multiple teachers, and a Teacher may have multiple students. In this case, we need a relational model to act as an intermediary model.
Relational models are not supposed to have any details. They only contain references to other models, which is their primary key. A composite key that aggregates two or more foreign keys within it. Let's take a look at an example:
There's a situation where a model connects multiple entities but also carries its data. I highly advise against following that path to maintain purity in your system design and control the complexity of your models. However, this approach is sometimes a necessary option to proceed with a specific implementation or business flow. In this case, we can propose a hybrid model that can carry particular details about the relationship between two independent entities.
A hybrid model can describe the detachment between two entities in a many-to-many relationship in a soft-delete scenario. Here's an example of a hybrid model that can occur in reality. Let's assume a group member does not want to be a part of a particular group anymore. We consider their group membership as Deactivated with a reason attached without actually deleting the record. Here's what it would that look like:
Hybrid models combine secondary models in the way they reference Primary models. They implement a relational nature in allowing multiple entities to relate to each other without exclusivity. In a non-relational data model, the referencing integrity may become looser, given the linear nature of that schema.
Operational models mainly target the simulation aspect of any software system. Think about all the primitive, complex, and exposure operations a simple scenario could require for a successful simulation to be implemented. Let's assume we are trying to solve a problem where we can simplify student registrations in some schools. The registration process will require some simulation to add these students' information into a computerized system.
Operational models will handle the entire process's exposure, processing, and integration by offering services that offer APIs/UIs to enter, post, add, and insert/persist students' information into some schooling systems.
The Standard focuses heavily on operational models because they represent the core of any system in terms of business flows. Operational models are also where most development and design resources go in any software development effort. Operational models can be broken into three main categories: Integration, Processing, and Exposure.
Let's talk about the operational models here.
Integration operational models' primary responsibility is to connect any existing system with external resources, which can be localized to the system's environment, like reading the current date or time, or remote, like calling an external API or persisting data in some database.
We call these integration models Brokers. They play a liaison role between processing operational models and external systems. Here's an example:
public partial class ApiBroker
{
public async ValueTask<Student> PostStudentAsync(Student student) =>
this.apiBroker.PostAsync<Student>(student, url);
}The integration model above offers the capability to call an external API while abstracting the configuration details away from the processing operational models.
Like any other operational model type, they don't hold data but instead, use private class members and constants to share internal data across their public and private methods. The ApiBroker here as a model represents a simulation of integration with an external system.
In upcoming chapters, we will discuss Brokers extensively to clarify the rules and guidelines for developing brokers with external resources or systems.
Processing models are the holders of all business-specific simulations, such as student registrations, requesting a new library card, or retrieving student information based on specific criteria. Processing models can be either primitive/foundational, high-order/processing, or advanced/orchestrators.
Processing models, in general, either rely on integration models or self-relying like computational processing services or rely on each other.
Here's an example of a simple foundational/primitive service:
public partial class StudentService : IStudentService
{
private readonly IStorageBroker storageBroker;
...
public async ValueTask<Student> AddStudentAsync(Student student) =>
await this.storageBroker.InsertStudentAsync(student);
}A higher-order service would do/look as follows:
public partial class StudentProcessingService : IStudentProcessingService
{
private readonly IStudentService studentService;
...
public async ValueTask<Student> UpsertStudentAsync(Student student)
{
....
Student maybeStudent = await this.studentService
.RetrieveStudentByIdAsync(student.Id);
return maybeStudent switch
{
null => await this.studentService.AddStudentAsync(student),
_ => await this.studentService.ModifyStudentAsync(student)
}
}
}More advanced orchestration-type services would combine multiple processing or foundational services as follows:
public partial class StudentOrchestrationService : IStudentOrchestrationService
{
private readonly IStudentProcessingService studentProcessingService;
private readonly IStudentLibraryCardProcessingService studentLibraryCardProcessingService;
...
public async ValueTask<Student> RegisterStudentAsync(Student student)
{
....
Student upsertedStudent = await this.studentProcessingService
.UpsertStudentAsync(student);
...
await this.studentLibraryCardProcessingService.AddStudentLibraryCardAsync(studentLibraryCard);
}
}In general, operational models are only concerned with the nature of simulation or processing of specific data carrier models; they are not concerned with holding data or retaining a status. In general, operational models are stateless in that they don't retain any details that went through them other than delegating logging for observability and monitoring purposes.
Exposure models handle the HMI in all scenarios where humans and systems interact. They could be simple RESTful APIs and SDKs or just UIs like in web, mobile, or desktop applications, including command-line-based systems/terminals.
Exposure operational models are like the integration models; they allow the outside world to interact with your system. They sit on the other end of any system and are responsible for routing every request, communication, or call to the proper operational models. Exposure models never communicate directly with integration models and don't have any configuration other than their dependencies injected through their constructors.
Exposure models may have their language in terms of operations; for instance, an integration model might use a language like InsertStudent, while an exposure model for an API endpoint would use a language like PostStudent to express the same operation in an exposure context.
Here's an example of exposure models:
public class StudentsController
{
private readonly IStudentOrchestrationService studentOrchestrationService;
[HttpPost]
public async ValueTask<ActionResult<Student>> PostStudentAsync(Student student)
{
Student registeredStudent = await this.studentOrchestrationService
.RegisterStudentAsync(student);
return Ok(registeredStudent);
}
}The above model exposes an API endpoint for RESTful communication to allow students to be registered into a schooling system. We will further discuss the types of exposure models based on the context and the systems they are implemented within.
The last type of model in any system is the configuration model. It can represent the entry point into a system, register dependencies for any system, or act as middleware to route URLs into their respective functions within an exposure model.
Configuration models usually appear at the beginning of a system's launch, handling incoming and outgoing communications or underlying system operations like memory caching, thread management, etc.
In a simple API application, you may see models that look like this:
public class Startup
{
public void ConfigureServices(IServices services)
{
services.AddTransient<IStorageBroker, StorageBroker>();
services.AddOAuth();
}
}As you can see from the code snippet above, the configuration model Startup offers capabilities to handle dependency injection-based registration of contracts to their concrete implementations. They may handle adding security or setting up a middleware pipeline. Configuration models are technology-specific. They may differ from a Play framework in Scala to a Spring or Flex in Python or Java. We will outline high-level rules according to The Standard for configuration models, but we will not dive deeper into the details of implementing any of them.
The simulation aspect of software engineering is our ability to resemble the interactions to and from the models. For instance, in the grocery store example, a simulation would be the act of selling the item. Selling the item requires multiple modifications to the item in terms of deducting the count of the available items and reordering the items left based on the most valuable available item.
We can describe the simulation process as illustrating the relationships between models, which are programmed as functions, methods, or routines; these terms all mean the same thing. If we have a software service that is responsible for item sales, a simulation process will look like this:
public class SaleService
{
public void Sell(Item item) => Items.Remove(item);
}In the example above, we have a model called SaleService that offers functionality to simulate the sales process in the real world on a model of an item. And this is how you describe everything in object-oriented programming. Everything is an object (from a model), and these objects interact with each other (simulation).
Object interaction, in general, can be observed in three different types. A model is taking an action on another model. For instance, the SaleService is executing an action of Sell on an Item model. That's a model interacting with another model. In the very same example, a simulation could be something happening to the model from another model, such as the Item in the example above. The last type of simulation is a model interacting with itself, such as models that self-dispose once their purpose is achieved, as they are no longer needed, so they self-destruct.
The simulation process is the third and last aspect of software engineering. We will explore it deeply when we discuss brokers, services, and exposers to illustrate how industrial software's modeling and simulation process happens.
If we consider purposing to be the domain or the framework in which models interact, then the following illustration should simplify and convey the picture a bit clearer:
It's important to understand that computer software can serve multiple purposes. Computer software can interact with other software that shares common purposes. The purpose of the software becomes the model, and the integrations become the simulations in that aspect. Here's a 10,000 feet example:
The complexity of any large system can be broken into smaller problems if the single-purpose or single-responsibility aspect is enforced for each subsystem. Modern software architectures call this granularity and modularization, which we will discuss briefly throughout the architecture aspect of The Standard.