gem-hunter
Language: TypeScript
Practical microservices, built with Node.Js, CQRS, Vertical Slice Architecture, Event-Driven Architecture, Postgres, RabbitMQ, Express and the latest technologies.
A practical and imaginary microservices for implementing an infrastructure for up and running distributed system with the latest technology and architecture like Vertical Slice Architecture, Event Driven Architecture, CQRS, Postgres, RabbitMQ and Express in Node.Js. 🚀
You can find
Nestjsported of this project in this link: 🔗 booking-microservices-nestjs
Vertical Slice Architecture for architecture level.Rabbitmq on top of amqp for Event Driven Architecture between our microservices.Data Centric Architecture based on CRUD in all Services.Rest for internal communication between our microservices with axios.CQRS implementation with MediatrJs internal library.Express for web framework.Postgres for database level with typeorm.tsyringe for handling dependency injection.Passport for authentication and authorization, base on JWT.OpenTelemetry for distributed tracing top of Jaeger and Zipkin.OpenTelemetry for monitoring top of Prometteuse and Grafana.Joi for validation input in our handlers and endpoints.dotenv for configuration management.Unit Testing for testing small units and mocking our dependencies with Jest.End-To-End Testing and Integration Testing for testing features with all dependencies using testcontainers.winston and 'morgan' for logging.swagger-ui-express and tsoa for generate api documentation automatically.ts-mapper for mapping our objects.Problem Details standard for readable details of errors.eslint and prettier for formatting of our code.Docker-Compose for our deployment mechanism.MongoDB for read side with mongoose.Domain Driven Design (DDD) to implement all business processes in microservices.Inbox Pattern for ensuring message idempotency for receiver and Exactly once Delivery.Outbox Pattern for ensuring no message is lost and there is at At Least One Delivery.🌀This project is a work in progress, new features will be added over time.🌀
I will try to register future goals and additions in the Issues section of this repository.
High-level plan is represented in the table
| Feature | Status |
|---|---|
| Identity Service | Completed ✔️ |
| Flight Service | Completed ✔️ |
| Passenger Service | Completed ✔️ |
| Booking Service | Completed ✔️ |
| Building Blocks | Completed ✔️ |
microsoft/TypeScript - TypeScript is a language for application-scale JavaScript.expressjs/express - Fast, unopinionated, minimalist web framework for Node.jstypeorm/typeorm - TypeORM is highly influenced by other ORMs, such as Hibernate, Doctrine and Entity Frameworkamqp-node/amqplib - A library for making AMQP 0-9-1 clients for Node.JSmicrosoft/tsyringe - A lightweight dependency injection container for TypeScript/JavaScript for constructor injectionjaredhanson/passport - Passport is Express-compatible authentication middleware for Node.jsopen-telemetry/opentelemetry-js - A framework for collecting traces, metrics, and logs from applicationssiimon/prom-client - A prometheus client for Node.js that supports histogram, summaries, gauges and countersscottie1984/swagger-ui-express - Serve auto-generated swagger-ui generated API docs from expresslukeautry/tsoa - OpenAPI-compliant REST APIs using TypeScript and Nodeaxios/axios - Promise based HTTP client for the browser and node.jsmotdotla/dotenv - Dotenv is a zero-dependency module that loads environment variables from a .envPDMLab/http-problem-details - This library implements HTTP Problem details (RFC 7807) for HTTP APIshelmetjs/helmet - Helmet helps secure Express apps by setting HTTP response headershapijs/joi - The most powerful schema description language and data validator for JavaScriptexpressjs/morgan - HTTP request logger middleware for node.jswinstonjs/winston - A logger for just about everythingjestjs/jest - A javascript framework for testingtestcontainers/testcontainers-node - A library to support tests with throwaway instances of Docker containersfaker-js/faker - Generate massive amounts of fake (but realistic) data for testing and developmentflorinn/typemoq - Simple mocking library for JavaScript targeting TypeScript developmentladjs/supertest - High-level abstraction for testing HTTPeslint/eslint - ESLint is a tool for identifying and reporting on patterns found in ECMAScript/JavaScript codeprettier/prettier - Opinionated Code Formattervercel/async-retry - Retrying made simple, easy, and asyncMarluanEspiritusanto/ts-mapper - A mapper for mapping one object to anotherIdentity Service: The Identity Service is a bounded context for the authentication and authorization of users using passport. This service is responsible for creating new users and their corresponding roles and permissions using Jwt for authentication and authorization.
Flight Service: The Flight Service is a bounded context CRUD service to handle flight related operations.
Passenger Service: The Passenger Service is a bounded context for managing passenger information, tracking activities and subscribing to get notification for out of stock products.
Booking Service: The Booking Service is a bounded context for managing all operation related to booking ticket.
In this project I used a mix of clean architecture, vertical slice architecture and I used feature folder structure to structure my files.
Each microservice has its dependencies such as databases, files etc. Each microservice is decoupled from other microservices and developed and deployed separately. Microservices talk to each other with Rest or gRPC for synchronous calls and use RabbitMq or Kafka for asynchronous calls.
We have a separate microservice Identity for authentication and authorization of each request. Once signed-in users are issued a JWT token. This token is used by other microservices to validate the user, read claims and allow access to authorized/role specific endpoints.
I used RabbitMQ as my MessageBroker for async communication between microservices using the eventual consistency mechanism. Each microservice uses amqp to interface with RabbitMQ providing, messaging, availability, reliability, etc.
Microservices are event based which means they can publish and/or subscribe to any events occurring in the setup. By using this approach for communicating between services, each microservice does not need to know about the other services or handle errors occurred in other microservices.
I treat each request as a distinct use case or slice, encapsulating and grouping all concerns from front-end to back.
When adding or changing a feature in an application in n-tire architecture, we are typically touching many "layers" in an application. We are changing the user interface, adding fields to models, modifying validation, and so on. Instead of coupling across a layer, we couple vertically along a slice. We minimize coupling between slices, and maximize coupling in a slice.
With this approach, each of our vertical slices can decide for itself how to best fulfill the request. New features only add code, we're not changing shared code and worrying about side effects.
Instead of grouping related action methods in one controller, as found in traditional ASP.net controllers, I used the REPR pattern. Each action gets its own small endpoint, consisting of a route, the action, and an IMediator instance (see MediatR). The request is passed to the IMediator instance, routed through a Mediatr pipeline where custom middleware can log, validate and intercept requests. The request is then handled by a request specific IRequestHandler which performs business logic before returning the result.
The use of the mediator pattern in my controllers creates clean and thin controllers. By separating action logic into individual handlers we support the Single Responsibility Principle and Don't Repeat Yourself principles, this is because traditional controllers tend to become bloated with large action methods and several injected Services only being used by a few methods.
I used CQRS to decompose my features into small parts that makes our application:
Using the CQRS pattern, we cut each business functionality into vertical slices, for each of these slices we group classes (see technical folders structure) specific to that feature together (command, handlers, infrastructure, repository, controllers, etc). In our CQRS pattern each command/query handler is a separate slice. This is where you can reduce coupling between layers. Each handler can be a separated code unit, even copy/pasted. Thanks to that, we can tune down the specific method to not follow general conventions (e.g. use custom SQL query or even different storage). In a traditional layered architecture, when we change the core generic mechanism in one layer, it can impact all methods.
Note: For easy using of migrations commands in typeorm, I add some scripts in
package.jsonand base on these scripts we can use below commands to generate and run migrations easily.
For generating a new migration use this command in the root of each microservice:
npm run migration:generate -- src/data/migrations/new-migration-name
Also for running migration use this command in the root of each microservice:
npm run migration:run
Use Docker Compose to run the whole application stack locally:
cp .env.example .env docker compose up -d --build
To inspect the running stack:
docker compose ps docker compose logs -f
To stop everything:
docker compose down
The Kubernetes manifests live in deployments/kubernetes.
Build the service images first:
docker build -t booking-microservices-expressjs-identity:latest -f src/identity/Dockerfile src docker build -t booking-microservices-expressjs-flight:latest -f src/flight/Dockerfile src docker build -t booking-microservices-expressjs-passenger:latest -f src/passenger/Dockerfile src docker build -t booking-microservices-expressjs-booking:latest -f src/booking/Dockerfile src
Then apply the Kubernetes manifests:
kubectl apply -k deployments/kubernetes
Useful checks:
kubectl get pods -n booking kubectl get svc -n booking kubectl get ingress -n booking
If you are using kind or another local cluster, load the locally built images into the cluster before applying the manifests.
To build each microservice, run this command in the root directory of each microservice where the package.json file is located:
npm run build
To run each microservice, run this command in the root of the microservice where package.json is located:
npm run dev
To test each microservice, run this command in the root directory of the microservice where the package.json file is located:
npm testEach microservice has a Swagger OpenAPI. Browse to /swagger for a list of endpoints.
As part of API testing, I created the booking.rest file which can be run with the REST Client VSCode plugin.
If you like my work, feel free to:
Thanks a bunch for supporting me!
Thanks to all contributors, you're awesome and this wouldn't be possible without you! The goal is to build a categorized, community-driven collection of very well-known resources.
Please follow this contribution guideline to submit a pull request or create the issue.
This project is made available under the MIT license. See LICENSE for details.