Notes for a course at Linnaeus University: https://coursepress.lnu.se/kurs/serverbaserad-webbprogrammering/
After the course you should be able to:
The course is divided into three parts
Recommended literature:
Node.js is not a web server nor a framework.
V8, Google chrome javascript engine, an event loop, and a low-level I/O API (libuv)
Small modules, doing one thing well, is the philosophy.
Single threaded event-driven architecture.
Event-driven programming:
Blocking or CPU-intensive code is not suitable for Node.js.
The cluster module allows easy creation of child processes that all share server ports.
npm install # installs all dependencies in package.json
yarn is an alternative to npm.
npm install mocha --save-dev
Use npx to execute a module (and install it temporarily)
Semantic Versioning (semver) major.minor.patch. Use * for latest.
The package-lock.json file describes the dependency tree and should be committed.
Different server services conventionally use certain ports.
Web servers are software that handles and understands HTTP/HTTPS request/response. Listen default on port 80 (HTTP) or port 443 (HTTPS)
Apache:
Nginx is a lightweight server commonly used as proxy or load balancer.
const http = require('http');
const PORT = 8080;
http.createServer((req, res) => {
res.writeHead(200);
res.end('Hello world\n');
}).listen(PORT);
Different kind of servers in a web environment
Domain name servers (DNS) translate domain names to IPs like 194.47.110.87 and 2001:6b0:52:4000::5:5
The browser sends a TCP/IP packet, the server sends back a reply for example on port 62365.
Stateless, securityless.
Internet Engineering Task Force (IETF) and the World Wide Web Consortium (W3C)
HTTP messages have headers.
Status codes:
https://en.wikipedia.org/wiki/List_of_HTTP_header_fields
The server should use gzip on text files > 1-2 Kb
Do not trust client ID, nor server ID, neither a human. Filter out crawlers in logs.
Set-Cookie: PHPSESSID=1p0sptqdupf47lefnti1j1fg40 Path=/admin
HTTP/2
const http = require('http')
const server = http.createServer((req, res) => {
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world\n')
})
server.listen(8000, () => console.log('Server running at http://localhost:8000/'))
http.createServer((req, res) => {
const path = req.url.replace(/\/?(?:\?.*)?$/, '').toLowerCase()
switch (path) {
case '':
res.setHeader('Content-Type', 'text/plain')
res.end('Homepage')
break
case '/about':
res.setHeader('Content-Type', 'text/plain')
res.end('About')
break
default:
res.setHeader('Content-Type', 'text/plain')
res.statusCode = 404
res.end('Not Found')
break
}
}).listen(8000, () => console.log('Server running at http://localhost:8000/'))
MEAN full stack JavaScript: MongoDB, Express, Angular, Node.js
Handlebars view engine.
Backend frameworks:
Express is inspired by Sinatra, a web application framework in Ruby, and intertwined with Connect, a pluggable Node module that can handle web requests (a.k.a. "middleware")
const express = require('express')
const app = express()
app.get('/', (req, res) => res.send('Homepage'))
app.get('/about', (req, res) => res.send('About'))
app.listen(8000, () => console.log('Server running at http://localhost:8000/'))
The directory structure can be
Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware function in the application’s request-response cycle.
router.get and router.post and app.use('/home', require('./routes/homeRouter'))
Controller functions render an HTML page using a template engine, res.render(view [, locals]) renders a view and sends the HTML string to the client.
Template engines: Pug, Mustache, EJS and express-hbs
MongoDB and Mongoose. Web sessions and flash messages.
Create, read, update and delete (CRUD) are the four basic function of persistent storage.
How to store application data at the server side?
Relational database management systems (RDBMS) organizes data into tables with columns and rows with Structured Query Language (SQL).
NoSQL databases like MongoDB and Firebase are like big maps where incomplete documents can be easily stored and extended during development without time consuming database migrations.
MongoDB is open-source and derives from the word humongous. Stores JSON-like documents.
The mongoose.connect method returns a Promise, mongodb+srv://
Close the Mongoose connection when the Node process ends.
const shoeSchema = new mongoose.Schema({
name: { type: String, required: true},
size: { type: Number, required: true, min: 15, max: 47 }
})
The lowercase name of the collection will automatically be the plural version of the model's name.
const Shoe = new mongoose.model('Shoe', shoeSchema)
const myShoe = new Shoe();
The module (Task.js) resides in the models directory.
router.get('/:id/edit', tasksController.edit)
router.post'/:id/update', tasksController.update)
The captured values are populated in the req.params object, with the name of the route parameter specified in the path as their respective keys.
const task = new Task({
description: req.body.description,
done: req.body.done
})
await task.save()
Use Task.find, and pass an empty object, to find all documents in a collection. It's good practice to transform the documents into anonymous objects before passing the data to the view.
Use Task.updateOne (deleteOne) with { _id: req.body.id } to update a document in the database.
A session cookie keeps users logged in.
Redis is an in-memory database alternative to MongoDB.
The default server-side session storage MemoryStorage is not for production.
Session variables are just properties of the request object's session property: req.session.name = 'Ada'
To avoid "double posting" if a form page is refreshed return a redirect command instead of a view directly, the Post/Redirect/Get (PRG) pattern.
Whenever you redirect someone on your website it is a good idea to use a flash message to let them know that what they just did worked or not. The flash message should survive only a round trip. Use a session variable to save the message and delete the message on the next request.
req.session.flash = { type: 'success', text: 'The requested action was completed.' }
Authentication and authorization.
Use cookie session id (stored on server), JWT (not stored) or a third-party like OpenId Connect or SAML.
If authorization fails 403 is the default status code however for security one can reply with 404 (or 418) to hide the fact that the server has the requested resource.
Passwords should be stored in an hashed format using strong cryptographic algorithms (Argon2, bcrypt, scrypt). Prevent rainbow table attacks with salt.
It is not sufficient to simply hide the delete button at the client from a user that are not allowed to delete something at the server. It is safer to prevent access by default and override any requests that do not require permission.
Salt and hash password using bcryptjs.
Applying the principle of separation of concerns is a good idea by adding static methods to the Mongoose schema to register users.
const userSchema = new mongoose.Schema({
username: {
type: String,
required: true,
unique: true
},
password: {
type: String,
required: true,
minlength: [10, 'The password must contain 10 characters.']
}
}, {
timestamps: true,
versionKey: false
})
Using a pre hook you can execute code after the validation and before the data is written to the database. You must call .pre before compiling the model:
userSchema.pre('save', async function () {
this.password = await bcrypt.hash(this.password, 8)
})
userSchema.statics.authenticate = async function (username, password) {
const user = await this.findOne({ username })
if (!user || !(await bcrypt.compare(password, user.password))) {
throw new Error('Invalid login attempt.')
}
return user
}
When defining a route you can provide multiple callbacks. You can use this mechanism to authorize the requested resource.
The application must always assume that all input is potentially malicious
Open Web Application Security Project
Web Security Testing Guide (WSTG)
XSS, Cross-Site Scripting. The attacker get the application to send unsanitized data (code) to client. Attackers can execute script on client to: Steal cookies (Session hijacking) and do key logging or phishing (Fake login)
Always validate/sanitize user submitted text, FIEO (filter input, escape output), encode characters like < and > etc.
https://github.com/YahooArchive/xss-filters
CSRF, Cross-Site Request Forgery (pronounced sea-surf). Forces a client to make an unwanted request to a web application in which the user currently is authenticated.
Protect with Synchronized Token pattern (STP)
<form action="/account" method="post">
<input type="hidden" name="_csrf" value="HvtsC1Ka-yq1Q2KPAu_Yh_H8F4vJEYfMIlBQ" />
</form>
https://github.com/expressjs/csurf
Injections: Manipulate the query to the database through HTTP requests
Handle all input as strings
Third party modules can be security issues, npm audit scans for vulnerabilities and automatically runs with npm install. Snyk.io has more features.
TLS(SSL) + HTTP = HTTPS. Always use HTTPS in production.
In development we can create our own self-signed certificate
Domain Validation (DV), Organization Validation (OV), Extended Validation (EV)
Asymmetric (public-key encryption)
A good article about different types of XSS attacks
OWASP, The Open Web Application Security Project, produces (every three year) a report presenting the 10 most common attacks against web application. This is a must read for every web developer. You should read the report to point A10 (page 16).
Web of things: Devices communicating in realtime
Examination 3: Create an application that checks issues in your repository. Fetch issue with Web API, monitor events with web hooks. OAuth.
"Flash Player has been deprecated and has an official end-of-life at the end of 2020"
Comet (umbrella term): Persistent HTTP connections, Domain sharding, Long-lived hidden iframe, XHR long polling, XHR streaming.
Server-sent events: Not implemented by microsoft.
Go WebSockets!
Long polling is mostly only for compatibility with old browsers.
Websockets connect on HTTP(S), Upgrade to ws:// or wss://
Challenges with web sockets:
P2P (peer to peer): http://www.html5rocks.com/en/tutorials/webrtc/basics/
Web hooks, server to server: tell me when event A happens.
Real time fundamentals blog series.
Bringing your node.js web application to production
Bringing your work to the people
https://github.com/CS-LNU-Learning-Objects/linux
Process manager PM2
#!/bin/bash
echo "Generating self-signed certificates..."
mkdir -p ./config/sslcerts
openssl genrsa -out ./config/sslcerts/key.pem 4096
openssl req -new -key ./config/sslcerts/key.pem -out ./config/sslcerts/csr.pem
openssl x509 -req -days 365 -in ./config/sslcerts/csr.pem -signkey ./config/sslcerts/key.pem -out ./config/sslcerts/cert.pem
rm ./config/sslcerts/csr.pem
chmod 600 ./config/sslcerts/key.pem ./config/sslcerts/cert.pem
How To Set Up Automatic Deployment with Git with a VPS
Updated on 2020-08-07.