Final submission draft

TAPAS-Final/doc/architecture/decisions/0001-record-architecture-decisions.md

@@ -0,0 +1,19 @@
+# 1. Record architecture decisions
+
+Date: 2021-10-18
+
+## Status
+
+Accepted
+
+## Context
+
+We need to record the architectural decisions made on this project.
+
+## Decision
+
+We will use Architecture Decision Records, as [described by Michael Nygard](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions).
+
+## Consequences
+
+See Michael Nygard's article, linked above. For a lightweight ADR toolset, see Nat Pryce's [adr-tools](https://github.com/npryce/adr-tools).

TAPAS-Final/doc/architecture/decisions/0002-seperate-service-for-executor-pool.md

@@ -0,0 +1,29 @@
+# 2. Seperate service for Executor Pool
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+The Executor Pool has to keep track of which Executors are online and what task types they can execute. The Roster keeps track of tasks that need to be executed and assigns Executors to tasks. The Executor Pool functionalty could be implemented in a seperate service or as a part of the Roster.
+
+## Decision
+
+The executor pool will be implemented as a seperate service. 
+
+Most importantly, the Executor Pool and the Roster have quite different responsibilities and reasons to change. On one hand, the Executor Pool manages the Executors, and therefore it will change if we want to add functionality that impacts how Executors log onto the system and how we keep track of them. For example, if we want to check if an executor fulfills some requirements before logging on. On the other hand, the Roster manages the execution of tasks. TODO 
+
+Single responsibility 
+
+Potential Code volatility 
+
+Scalability - Roster needs much more scale
+
+## Consequences
+
+The one funcionalty that is duplicated is error handling when an executor disconnects nongracefully... 
+
+TODO

TAPAS-Final/doc/architecture/decisions/0003-seperate-service-for-roster.md

@@ -0,0 +1,21 @@
+# 3. Separate service for the Roster
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+The roster acts as an orchestrator for the system. It communicates directly with the task list, the executors, the executor pool, and the auction house. It handles the assignment of a task to a corresponding and available executor, keeps track of all the connections between tasks and executors, and communicates the status of tasks and executors to other services.
+
+## Decision
+
+The Roster domain will be its own service.
+The Roster service will be a central point in our application. It will have most of the workflow logic in it and will communicate with all the different services. Therefore, other services can focus on their business logic and be largely ignorant of the overall workflow.
+The code of the assignment will change more often than the code of the other services, thus having the assignment service split from the other makes it more deployable.
+
+## Consequences
+
+Having this system as its own service will reduce the fault tolerance because the assignment service can be the single point of failure. We can mitigate this risk by implementing (server) replication and/or having an event driven communication with persisting messages. Therefore, all other services can run independently, and the assignment service can recover from a crash. Additionally, we need to ensure a high level of interoperability, since the roster has to communicate with all other parts of the system.

TAPAS-Final/doc/architecture/decisions/0004-seperate-service-for-task-list.md

@@ -0,0 +1,20 @@
+# 4. Separate service for the Task List
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+Tasks are created in the task list, and the status of each task (created, assigned, executing, executed) is tracked in the task list as well. The task list mainly communicates with the roster so that tasks can get assigned and the roster will give the task list feedback about the tasks’ status.
+
+## Decision
+
+The task list will be its own service.
+The task list needs to scale based on the number of active users and the intensity of their activity at any time while the scaling of other parts of the system can be constrained by other factors.
+
+## Consequences
+
+Although having the task list as its own service might slightly increase the complexity of the system and decrease the testability, it also makes the system easier to deploy and protective of its data. However, to ensure that this data is always available and does not get lost, the task list needs to be able to recover all its data (the entire history of all tasks) in case it goes down.

TAPAS-Final/doc/architecture/decisions/0005-event-driven-communication.md

@@ -0,0 +1,20 @@
+# 5. Event driven communication
+
+Date: 2021-10-18
+
+## Status
+
+Superceded by [8. Switch to an event-driven microservices architecture](0008-switch-to-an-event-driven-microservices-architecture.md)
+
+## Context
+
+Services need to be able to communicate with each other. Services need to be scalable and therefore multiple services will need to get the same messages. Most of the processes are about responding to events that are happening throughout the system.
+
+## Decision
+
+We will use mainly event driven communication.
+
+## Consequences
+
+Event driven communication will help use to create a system which has high scalability and elasticity. Through persisting messages, we will also reach way higher fault tolerance and recoverability.
+Having an event driven communication, we can only guarantee eventual consistency.

TAPAS-Final/doc/architecture/decisions/0006-one-global-database-or-one-database-per-service.md

@@ -0,0 +1,20 @@
+# 6. One global database or one database per service
+
+Date: 2021-10-18
+
+## Status
+
+Accepted
+
+## Context
+
+We can have one database for all services or each Microservice can have its own database.
+
+## Decision
+
+Each Microservice will have its own database.
+The different services don’t need to store a lot of similar data. Therefore, we can have every Microservice handle its own data. This also gives the advantage that every Microservice owns its own data and is also responsible for it. (Data ownership, Data responsibility).
+
+## Consequences
+
+Having one database per Microservice will lead to eventual consistency. Having an event driven communication we can use event-based synchronisation to keep the data in sync between the services, thus the individual services don’t need to know about each other. To guarantee data consistency we can also use a pattern like sagas.

TAPAS-Final/doc/architecture/decisions/0007-seperate-service-for-auction-house.md

@@ -0,0 +1,22 @@
+# 7. Seperate service for Auction House
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+The auction house is the service that can connect to other groups’ auction houses. If there is a task whose task type does not match that of our executors, the auction house can start an auction where other groups can bid on doing the task for us. Moreover, it can also bid on other groups’ auctions.
+
+## Decision
+
+The auction house will be its own service.
+The auction house is the only part of our system that has external communication; therefore, it makes sense to have it as its own service, also to guarantee better deployability.
+The auction house does not scale directly based on the number of tasks, but only the proportion which needs external executors. Moreover, there could be limits on the number of auctions that could be started. Therefore, the auction house scales differently to other services.
+Moreover, having the auction house as its own service also improves the fault tolerance of our system.
+
+## Consequences
+
+Since the auction house will be a standalone service, we have to make sure that if it goes down, it can recover its data in some way (which auctions it has launched, which auctions it has placed bids on or even won, etc.). Even though the testability and latency of our system might worsen by having a separate service for the auction house, we can implement different kinds of communication for internal and external communication in a much easier way.

TAPAS-Final/doc/architecture/decisions/0008-switch-to-an-event-driven-microservices-architecture.md

@@ -0,0 +1,26 @@
+# 8. Switch to an event-driven microservices architecture
+
+Date: 2021-11-21
+
+## Status
+
+Proposed
+
+Supercedes [5. Event driven communication](0005-event-driven-communication.md) TODO Fix this. Should only supercede it if this has been accepted. This should also subercede 0013 - Microservice Architecture if accepted.
+
+## Context
+
+Our Tapas App is currently implemented based on a microservice architecture, where the services communicate synchronously via request-response. Each service encapsulates a different bounded context with different functional and non-functional requirements. Internal communication could also be done using asynchronous or event-driven communication.
+
+## Decision
+
+Pros:
+Scalability: Different services within the Tapas app are not always able to scale at the same rate. For example, we could have thousands of users adding printing tasks at the same time, but maybe we only have one printer. In this scenario we might want to scale the task-list service up to handle the creation load, but scaling up the printing executor operates on a different time-scale (i.e. adding a printer takes time). Moreover, we could have a lot of new tasks coming in, most of which can be executed internally. In this case we want to be able to scale up the task list but might not need to scale up the auction house. Event-driven communication would decrease the coupling of services. Consequently, the scalability of individual services would be enhanced as they no longer depend on the scalability of other services. This improves the apps overall scalability. Since scalability is one of the systems top 3 -ility, this seems quite important.
+Fault tolerance: Another of the systems top 3 -ilities is fault tolerance. We could have highly unstable IoT executors that fail often. This should not disrupt the system’s overall operation. The decoupling facilitated by event-driven, asynchronous, communication ensures that when individual services go down, the impact of other services is limited and once they go back up then can recover the systems state from persisted messages.
+Cons:
+Error handling, workflow control, and event timing:
+The aforementioned topics outline the drawbacks of event- driven architecture. These drawbacks can be mitigated by using an orchestrator (like we currently do with the roster) to orchestrate assignment of tasks, auctioning off tasks and error handling when executors fail. More research needed.
+
+## Consequences
+
+Consequences to be determined but would relate to the three concepts mentioned as cons.

TAPAS-Final/doc/architecture/decisions/0009-common-library-for-shared-code.md

@@ -0,0 +1,19 @@
+# 9. common library for shared code
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+The numerous services that make up the Tapas app all have common, non-domain specific, functionality that can be shared via a common library, or be replicated in each service. 
+
+## Decision
+
+Use a common code library for shared code which does not change frequently, but if it would be changed, would need to be changed everywhere. 
+
+## Consequences
+
+Changes in the common code will most likely require multiple services to be redeployed. However, those services would most likely have to have been changed individually and redepolyed anyways. Another consequence is that versioning becomes more complicated. 

TAPAS-Final/doc/architecture/decisions/0010-executor-base-library.md

@@ -0,0 +1,20 @@
+# 10. executor base library
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+Executors all use the same logic to communicate with other services. This means that their code base is near identical except for the code that implements their specific task execution. The code that implements the executor's shared logic can either be implemented by a shared library, or replicated across all executors.
+
+## Decision
+
+Since all executors use the same logic to communicate with other services, any change to this logic would have to be made for every executor. We will therefore use a shared library for executors, called executor-base. The library includes all the shared logic which every executor needs. 
+Having this shared logic in a separate library makes it easy to change the common logic at one place. The code sharing happens at compile time, which reduces the chance of runtime errors, compared to other code sharing approaches. If other people need to create executors, they can just reference the executor-base library and implement the actual execution part. Therefore they don’t need to worry about the the connection implementations to the Tapas system. 
+
+## Consequences
+
+It becomes easier to change the way that the executors communicate with the rest of the system. Moreover, changes are less risky as they only need to be implemented once. It also becomes easier for other teams within the organisation to create executors as they can use the executor-base to implement the shared logic. However, using a shared library will increase the complexity of the executors. Also there needs to be a clear way to use proper versioning. Lastly, by using this library we might be making assumptions for future executors that might not hold. For example, if we want to create more lightweight executors for IoT devices we might need to create a separate base package (if the current one becomes too fat), so that the executors can stay lightweight and don’t implement unused code.

TAPAS-Final/doc/architecture/decisions/0011-seperation-of-common-and-executor-base-library.md

@@ -0,0 +1,19 @@
+# 11. seperation of common and executor-base library
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+We have two code sharing libraries, the executor-base and the common library. The executor-base implements shared logic that all executors need, but other services don't. The common library has much more wide reaching implementations, such as the implementation of the SelfValidating class. These could form a single common library, or two seperate common libraries. 
+
+## Decision
+
+There will be a separate library for common and executor-base. The libraries share different type of code, and have different reasons to change. It would not make sense to have the shared code from executors in every other service which needs access to other shared code. Services that use the code in the common library should not need to be dependent in any way on the executor-base.
+
+## Consequences
+
+Changes impact fewer services. However, this decision will increase the number of service dependencies and therefore increase complexity in managing those dependencies.  

TAPAS-Final/doc/architecture/decisions/0012-separate-service-for-each-executor.md

@@ -0,0 +1,27 @@
+# 12. separate service for each executor
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+Executors must receive tasks of different types and execute them. The executors could either all be implemented within one service or as multiple services, one for each type of executor.
+
+## Decision
+
+We will have a seperate service for each type of executor.
+
+Firstly, execution time differs significantly between task types. Therefore, having seperate services will allow the executos to scale differently based on their tasks' specific needs.
+
+Secondly, the systems functioning should not be disrupted in case an Executor fails. Having each type of executor in a seperate service will increase fault tolerance in this regard.
+
+Lastly, extensibility is one of the systems most important non-functional requirement and providers of executors need to be able to easily add executors to the executor pool. These factors are also positively impacted by having seperate services.
+
+There should not be any shared data between the executors. Additionally, there should be no flow of information between them. Thus, there should be no issues due to workflow and data concerns due to this decision.
+
+## Consequences
+
+Executors share a lot of functionality when it comes to connecting to the rest of the system. Therefore, this decision means that we will either have to duplicate the code that implements the common functionality, or we have to have a way to share the code (e.g. through a common library)

TAPAS-Final/doc/architecture/decisions/0012-seperate-service-for-each-executor.md

@@ -0,0 +1,27 @@
+# 12. seperate service for each executor
+
+Date: 2021-11-21
+
+## Status
+
+Accepted
+
+## Context
+
+Executors must receive tasks of different types and execute them. The executors could either all be implemented within one service or as multiple services, one for each type of executor.
+
+## Decision
+
+We will have a seperate service for each type of executor.
+
+Firstly, execution time differs significantly between task types. Therefore, having seperate services will allow the executos to scale differently based on their tasks' specific needs.
+
+Secondly, the systems functioning should not be disrupted in case an Executor fails. Having each type of executor in a seperate service will increase fault tolerance in this regard.
+
+Lastly, extensibilty is one of the systems most important non-functional requirement and providers of executors need to be able to easily add executors to the executor pool. These factors are also positively impacted by having seperate services. 
+
+There should not be any shared data between the executors. Additionally, there should be no flow of information between them. Thus, there should be no issues due to workflow and data concerns due to this decision.
+
+## Consequences
+
+Executors share a lot of functionality when it comes to connecting to the rest of the system. Therefore, this decision means that we will either have to duplicate the code that implements the common functionality, or we have to have a way to share the code (e.g. through a common library)

TAPAS-Final/doc/architecture/decisions/0013-microservice-architecture.md

@@ -0,0 +1,23 @@
+# 13. microservice architecture
+
+Date: 2021-12-02
+
+## Status
+
+Accepted
+
+## Context
+
+The system is made up of five distinct bounded contexts, namely the Task Domain, the Roster Domain, the Executor Pool Domain, the Executor Domain, and the Auction Domain. The way that these bounded contexts function together to fulfil the system requirements can be based on many different architectures. (Feedback needed. Should we name specific 'next-best' alternative architectures to compare, or just leave it as is since technically all architectures were considered)
+
+## Decision
+
+The system will follow the Microservice architecture. 
+
+Scalability and fault tolerance are two of the systems top 3 -ilities. Moreover, elasticity and evolvability are two of the systems other main -ilities. These are all non-functional requirements that the Microservice architecture excels at.
+
+We do not expect to have a single monolithic database, so this is not a concern.
+
+## Consequences
+
+There is a considerable amount of communication between bounded contexts. This could cause responsiveness and performance issues due to added latency. This could therefore mean we would need to use asynchronous REST calls or publish-subscribe communication to mitigate these issues as much of the communication does not have to be synchronous. 

TAPAS-Final/doc/architecture/decisions/0014-data-ownership.md

@@ -0,0 +1,19 @@
+# 14. data ownership
+
+Date: 2021-12-02
+
+## Status
+
+Accepted
+
+## Context
+
+The issue motivating this decision, and any context that influences or constrains the decision.
+
+## Decision
+
+The change that we're proposing or have agreed to implement.
+
+## Consequences
+
+What becomes easier or more difficult to do and any risks introduced by the change that will need to be mitigated.