Without any doubt Express is one of the most popular web frameworks out there. After its initial release on November 16, 2010, Express is still going strong with 50k+ stars on GitHub and being the base for a number of new web frameworks including Sails.js, NestJS, Feathers being the most popular ones.
Part of Express’s charm is its simplicity. It’s fast, unopinionated and minimal. In other words, Express provides you with a powerful middleware system good enough for building great applications and lets you run free. You can make an entire application within a single app.js
file or a robust monolith by replicating the MVC pattern and Express won’t complain. This level of freedom while not exclusive to Express (looking at you Koa ) is a blessing and a curse, because you never know the architecture you’re following is good enough or not in the long run.
In these situations, the Express application generator although helpful, is outdated and can not keep up with the needs of modern scalable web applications. Apart from this generator, there are a number of popular project boilerplates out there.
One of these boilerplates is sahat/hackathon-starter with 30k+ stars on GitHub and a huge set of features to get you started with. But what I dislike about this boilerplate is the fact that it’s too bloated for my needs. I mostly build APIs with Express and this boilerplate comes with a hefty view layer. I surely can cut that off but other parts of the boilerplate are also well suited for full-stack applications rather than REST APIs.
The second one and the one I like the most is santiq/bulletproof-nodejs boilerplate with 3k+ stars on GitHub. Unlike the previous one, this boilerplate is much lighter and very well suited for building APIs. It’s also written in TypeScript which is a plus and follows a number of design patterns that I like. Although a robust boilerplate I have two issues with this. They are as follows –
- I don’t do TypeScript that often hence the DI container implementation won’t work well for me.
- The code in this boilerplate is structured by technical roles of the files instead of components.
If you want to learn more about this architecture, you may read this blog post.
After looking for a boilerplate that suits me and building few APIs myself, I’ve finally come up with a rock solid architecture that in my opinion strikes the right balance between features, best-practices and simplicity.
fhsinchy/express-mongo-api-boilerplate
In this article I’ll discuss the goods, bads and uglies of express application architecture as well as how I came up with this architecture, why I think this is good and how you can use or extend this project for your needs.
Folder Structure
The top level folder structure is as follows –
. ├── docker-compose.yaml ├── LICENSE ├── Makefile ├── README.md └── src
The Makefile
contains common bash commands that I use for tasks like starting and stopping the containers, seeding data, seeing logs etc. The docker-compose.yaml
file contains definitions for the different services such as the database service, the API itself and a mongo-express service for easy database administration. Source code for the API lives inside src
. The top level structure of that directory is as follows –
. ├── app.js ├── bin ├── config ├── Dockerfile ├── Dockerfile.dev ├── Dockerfile.test ├── jest.config.js ├── log ├── nodemon.json ├── package.json ├── routes.js ├── seed.js └── seeds
Directory and file names are pretty self explanatory. The config
directory holds configuration for the database, cors, seeder, authentication and some common app oriented configuration.
The Dockerfile
, Dockerfile.dev
and Dockerfile.test
files are used for building the production, development and test images respectively.
Daily generated logs are stored inside log
directory. A log file for each day is generated using <project-name>-<date>.log
naming pattern.
The routes.js
file is responsible for registering different routes with the app
instance.
The seeds
directory contains database seeds and seed.js
is a simple seeder script that we’ll come back to later on.
The nodemon.json
and jest.config.js
files are configuration files for nodemon and Jest testing framework.
The “app” and “server” Instances
When I say app
instance and server
instance, what I’m actually referring to is –
/** * app instance initialization. */ const app = express(); /** * server instance initialization. */ const server = http.createServer(app);
For further explanation, we’ll first have to take a look at the app.js
and bin/www
files. The content of the app.js
file is as follows –
/** * Module dependencies. */ const cors = require('cors'); const { join } = require('path'); const logger = require('morgan'); const helmet = require('helmet'); require('pkginfo')(module, 'name'); const express = require('express'); const rfs = require('rotating-file-stream'); const { isCelebrate } = require('celebrate'); const cookieParser = require('cookie-parser'); const config = require('./config'); /** * app instance initialization. */ const app = express(); /** * Middleware registration. */ app.use(cors(config.cors)); app.use(helmet()); app.use(express.json()); app.use(cookieParser()); /** * Logger setup. */ app.use(logger('common')); app.use( logger('combined', { stream: rfs.createStream( `${module.exports.name}-${new Date() .toISOString() .replace(/T.*/, '') .split('-') .reverse() .join('-')}.log`, { interval: '1d', path: join(__dirname, 'log'), }, ), }), ); /** * Route registration. */ require('./routes')(app); /** * 404 handler. */ app.use((req, res, next) => { const err = new Error('Not Found!'); err.status = 404; next(err); }); /** * Error handler registration. */ app.use((err, req, res, next) => { const status = isCelebrate(err) ? 400 : err.status || 500; const message = config.app.env === 'production' && err.status === 500 ? 'Something Went Wrong!' : err.message; if (status === 500) console.log(err.stack); res.status(status).json({ status: status >= 500 ? 'error' : 'fail', message, }); }); module.exports = app;
As you can see, this file is used for bootstrapping and exporting the Express app
instance. This app then gets imported inside the bin/www
file. The content of the bin/www
file is as follows –
#!/usr/bin/env node /** * Module dependencies. */ const http = require('http'); const mongoose = require('mongoose'); const app = require('../app'); const config = require('../config'); /** * Get port from environment and store in Express. */ const { host } = config.app; const { port } = config.app; app.set('port', port); /** * Create HTTP server. */ const server = http.createServer(app); /** * ODM initialization. */ mongoose .connect(config.db.connectionString, config.db.connectionOptions) .catch((err) => console.log(err)); mongoose.connection.on('error', (err) => { console.log(err); }); /** * Listen on provided port, on all network interfaces. */ console.log(`app running -> ${host}:${port}`); server.listen(port);
This file is responsible for setting up mongoose
and spinning up the server
instance. Although this file should be responsible for the later task only, I had to put my mongoose
initialization code here to avoid cyclic dependency issues. I’ll surely find a better place to put this code.
Keeping the app
instance separate from the server
facilitates testing in isolation. The app
instance and server
instances can imported inside test files and tested without bumping into each other.
The route registration here is another story. I like my app.js
file frozen. Which means I don’t want this file to change now and then. That’s why I’ve moved the route registration logic to another file routes.js
. The content of the file is as follows –
module.exports = (app) => { app.get('/', (req, res) => { res.status(200).json({ error: false, message: 'Bonjour, mon ami', }); }); }
This file exports an arrow function that takes the app
instance as parameter. Route middleware are then attached to this instance. Call to this exported function can be seen in app.js
file –
// ... /** * Route registration. */ require('./routes')(app); // ...
This way I can keep the app.js
file away from frequent changes, and registering routes in a separate file keeps the code cleaner.
Components
A common pattern seen not only in Express but also in other platforms is to group code by their technical role instead of components. One example can be as follows –
. ├── app.js ├── bin ├── controllers ├── helpers ├── middleware ├── migrations ├── models └── tests
This is one of my older projects. As you can see, code is grouped according to their technical roles inside models
, migrations
, controllers
, middleware
and helpers
directories. Although it works fine for small projects, you’ll find these directories extremely cluttered as the project grows. At this moment, this code-base holds ~30 files inside each of those directories.
A better approach is to group files according to components. In an e-commerce application, possible components can be as follows –
- auth – handles authentication and authorization
- admin – handles administrative tasks
- cart – people put their stuff here
- shop – deals with indexing and showing the products
- inventory – handles stock management for the admins
As you can imaging the auth
component for example holds necessary route handlers, middleware and business logic to handle authentication, authorization features. The structure of this component can be as follows –
. ├── api ├── middleware ├── models └── services
The api
directory holds necessary logic for handling the HTTP requests. These can be treated as the controllers. The middleware
directory holds the middleware (duh) such as the authenticate
middleware responsible for guarding routes from unauthenticated access. Together, these two directories make up the web layer.
The models
directory holds the database models (schemas) and the services
directory is the service layer for this component.
The Web and Service Layers
It’s a common practice to divide web applications into three separate layers namely –
- Web Layer
- Data Access Layer
- Service Layer
The previously mentioned api
directory along with the middleware
directory inside a component can be treated as the web layer, responsible for transporting requests and responses. The data access layer is usually responsible for working with the database directly. But as we’re using mongoose
instead of working with the database directly, we can omit this layer.
The service layer is in my opinion the most important. It holds necessary business logic for performing various tasks such as registering a user in case of the auth
component.
To better understand this concept, let’s take a look at the content of the auth/api/routes/auth.js
file –
const { Router } = require('express'); const { celebrate, Joi } = require('celebrate'); const { User } = require('../../models'); const config = require('../../../config'); const { AuthService } = require('../../services'); const { authenticate } = require('../../middleware'); const router = Router(); const authService = new AuthService(User); module.exports = (routes) => { routes.use('/auth', router); router.post( '/register', celebrate({ body: Joi.object().keys({ name: Joi.string().trim().required(), email: Joi.string().email().trim().required(), password: Joi.string().required(), }), }), async (req, res, next) => { try { res.status(201).json({ status: 'success', message: 'User Registered!', data: { user: await authService.signup(req.body), }, }); } catch (err) { next(err); } }, ); router.post( '/login', celebrate({ body: Joi.object().keys({ email: Joi.string().email().trim().required(), password: Joi.string().required(), }), }), async (req, res, next) => { try { const { accessToken, refreshToken } = await authService.login(req.body); res.cookie('refreshToken', refreshToken, config.auth.refreshToken.cookie.options); res.status(200).json({ status: 'success', message: 'User Logged In!', accessToken, }); } catch (err) { next(err); } }, ); router.post('/logout', authenticate, (req, res) => { res.clearCookie('refreshToken'); res.status(200).json({ status: 'success', message: 'Logged Out!', }); }); };
This file is only responsible for transporting requests and responses as I’ve already mentioned and yes, I consider validating a part of the process. Many people put validation in a separate layer, but that seems like overengineering to me. The business logic necessary for performing the requested task is inside the AuthService
class exported from services/auth.js
file –
const bcrypt = require('bcrypt'); const jwt = require('jsonwebtoken'); const config = require('../../config'); module.exports = class AuthService { constructor(User) { this.User = User; } async signup(params) { if (await this.User.findOne({ email: params.email }).exec()) { const err = new Error('Email Already Taken!'); err.status = 400; throw err; } else { const user = await this.User.create({ name: params.name, email: params.email, password: await bcrypt.hash(params.password, 12), }); return { name: user.name, email: user.email, }; } } async login(params) { const user = await this.User.findOne({ email: params.email }).exec(); if (!user) { const err = new Error('Wrong Email!'); err.status = 400; throw err; } else if (await bcrypt.compare(params.password, user.password)) { const tokenPayload = { name: user.name, email: user.email, }; const accessToken = jwt.sign(tokenPayload, config.auth.accessToken.secret, { expiresIn: config.auth.accessToken.validity, }); const refreshToken = jwt.sign(tokenPayload, config.auth.refreshToken.secret, { expiresIn: config.auth.refreshToken.validity, }); return { accessToken, refreshToken, }; } else { const err = new Error('Wrong Password!'); err.status = 400; throw err; } } };
As you can see, the AuthService
class takes the User
model as a dependency. The actions inside the service receives the request parameters passed by the route handlers, performs necessary actions and returns the processed output. The web layer then returns the values to the client.
The beauty of this approach is that the business logic can now be called from anywhere, since it’s not a part of the web layer. All it requires is some dependencies, and it’s usable even from command line applications.
A good example is the seed.js
file, which is a DIY seeder implementation for mongoose
utilizing the service layer. I’ll write about this in another article.
These services can be tested in isolation as well without hitting the app
instance.
Closing Thoughts
The architecture of your application will always be dictated by your necessities. This boilerplate or architecture is by no means perfect. It’s something that I’ve been using in medium to large scale APIs (both REST and GrahphQL) for quite some time, and it hasn’t let me down even once. Look around the code-base, if you think it’s suitable for your use-case, leave a start and use this as template. If you think something can be improved, let me know or heck you can just contribute directly. That’s the beauty of open-source, isn’t it?
Hi Farhan
Just watched your video on FreeCodeCamp and it was fantastic. This post on Express is also very interesting.
Thanks a lot for checking out my course Avi. I hope to produce more in the future.
Hello Frahan,
It’s a great article on express architecture.
I was thinking of a few tweaks here and there, you might be interested in.
i.e. On ‘AuthService’ class:
For ‘signup’ method, that else is not necessary, since you are already throwing the error early. It’s just adding another level of indentation.
And for ‘login’ method, instead of “else if” you could just introduced another if with reversed “bcrypt.compare” (for password not matched) and throw that error early.
To my eyes, codes just look messy when there are many layers of indentation. (may be it’s just me)
But all in all, perfect article.
I really did enjoy reading it.
Thank you very much.
Hello Wafi. That’s an excellent suggestion. Feel free to make the changes in my github repo and I’d happy to merge them. It’s been a while since I last touched that codebase. Thanks again for the suggestion friend.