unhosted web apps

freedom from web 2.0's monopoly platforms

3. Setting up your personal server

(en Français)

Why you need a server

Last week we created an unhosted javascript/html/markdown editor that can run on a data URL, create other apps as data URLs, load files from the local file system, and also save them again. This doesn't however allow us to communicate with other devices, nor with other users, let alone people in other "web 2.0 worlds". At some point we will want to send and receive emails and chat messages, so to do those thing with an unhosted web app, we need a sort of proxy server.

There are three important restrictions of unhosted web apps compared to hosted web apps:

For these reasons, you cannot run your online life from unhosted web apps alone. You will need a personal server on the web. To run your own personal server on the web, there are initially four components you need to procure:

Shopping for the bits and bobs

The VPS is the most expensive one, and will cost you about 15 US dollars a month from for instance Rackspace. The domain name will cost you about 10 US dollars per year from for instance Gandi, and the TLS certificate you can get for free from StartCom, or for very little money from other suppliers as well. You could in theory set up your DNS hosting on your VPS, but usually your domain name registration will include free DNS hosting.

Unless you already have a domain name, you need to make an important choice at this point, namely choosing the domain name that will become your personal identity on the web. Most people have the same username in different web 2.0 worlds, for instance I often use 'michielbdejong'. When I decided it was time for me to create my own Indie Web presence, and went looking for a domain name a few months ago, 'michielbdejong.com' was still free, so I picked that. Likewise, you can probably find some domain name that is in some way related to usernames you already use elsewhere.

I registered my TLS certificate for 'apps.michielbdejong.com' because I aim to run an apps dashboard on there at some point, but you could also go for 'www.' or 'blog.' or whichever subdomain tickles your fancy. In any case, you get one subdomain plus the root domain for free, plus as many extra origins as you want on ports other than 443. So you cannot create endless subdomains, but you can host different kinds of content in subdirectories (like this blog that is hosted on /adventures on the root domain of unhosted.org), and you can get for instance https://michielbdejong.com:10001/ as an isolated javascript origin that will work on the same IP address and TLS certificate.

No Cookie Crew - Warning #1: Last week's tutorial could be followed entirely with cookies disabled. I am probably still the only person in the world who has all cookies disabled, but in case readers of this blog want to join the "No Cookie Crew", I will start indicating clearly where violations are required. Although Rackspace worked without cookies at the time of writing (they used URL-based sessions over https at the time, not anymore), StartCom uses client-side certificates in their wizard and other services are also likely to require you to white-list their cookies. In any case, you need to acquire three actual products here, which means you will need to white-list cookies from second-party e-commerce applications, and also corresponding control-panel applications which are hosted by each vendor. So that in itself is not a violation, but you will also probably need to use some desktop or hosted application to confirm your email address while signing up, and you will need something like an ssh and scp client for the next steps.

First run

Once you have your server running, point your domain name to it, and upload your TLS certificate to it. The first thing you always want to do when you ssh into a new server is to update it. On Debian this is done by typing:


apt-get update
apt-get upgrade

There are several ways to run a server, but here we will use nodejs, because it's fun and powerful. So follow the instructions on the nodejs website to install it onto your server. Nodejs comes with the 'node' executable that lets you execute javascript programs, as well as the 'npm' package manager, that gives you access to a whole universe of very useful high-quality libraries.

Your webserver

Once you have node working, you can use the example from the nodejitsu docs to set up your website; adapted here to make it serve your website over https:


var static = require('node-static'),
    http = require('http'),
    https = require('https'),
    fs = require('fs'),
    config = {
      contentDir: '/var/www',
      tlsDir: '/root/tls'
    };

http.createServer(function(req, res) {
  var domain = req.headers.host;
  req.on('end', function() {
    res.writeHead(302, {
      Location: 'https://'+domain+'/'+req.url.substring(1),
        'Access-Control-Allow-Origin': '*'
      });
    res.end('Location: https://'+domain+'/'+req.url.substring(1));
  });
}).listen(80);

var file = new(static.Server)(config.contentDir, {
  headers: {
    'Access-Control-Allow-Origin': '*'
  }
});

https.createServer({
  key: fs.readFileSync(config.tlsDir+'/tls.key'),
  cert: fs.readFileSync(config.tlsDir+'/tls.cert'),
  ca: fs.readFileSync(config.tlsDir+'/ca.pem')
}, function(req, res) {
  file.serve(req, res);
}).listen(443);

Here, tls.key and tls.cert are the secret and public parts of your TLS certificate, and the ca.pem is an extra StartCom chain certificate that you will need if you use a StartCom certificate.

Note that we also set up an http website, that redirects to your https website, and we have added CORS headers everywhere, to do our bit in helping break down the web's "Same Origin" walls.

Using 'forever' to start and stop server processes

Now that you have a script for a http server and a https server, upload them to your server, and then run:


npm install  node-static
npm install -g forever
forever start path/to/myWebServer.js
forever list

If all went well, you will now have a new log file in ~/.forever/. Whenever it gets too long, you can issue 'echo > ~/.forever/abcd.log' to truncate the log of forever process 'abcd'.

You can test that your http server redirects to your https website, and you can add some basic placeholder information to the data directory, or copy and paste your profile page from one of your existing web 2.0 identities onto there.

File sharing

One advantage of having your own website with support for TLS is that you can host files for other people there, and as long as your web server does not provide a way to find the file without knowing its URL, only people who know or guess a file's link, and people who have access to your server, will be able to retrieve the file. This includes employees of your VPS providers, as well as employees of any governments who exert power over that provider.

In theory, employees of your TLS certificate provider can also get access if they have man-in-the-middle access to your TCP traffic or on that of the person retrieving the file, or if they manage to poison DNS for your domain name, but that seem very remote possibilities, so if you trust your VPS provider and its government, or you host your server under your own physical control, and you have a good way to generate a URL that nobody will guess, then this is a pretty feasible way to send a file to someone. If you implement it without bugs, then it would be more secure than standard unencrypted email, for instance.

In the other direction, you can also let other people send files to you; just run something like the following script on your server:


var https = require('https'),
    fs = require('fs'),
    config = {
      tlsDir: '/root/tls',
      uploadDir: '/root/uploads',
      port: 3000
    },
    formidable = require('formidable'),
    

https.createServer({
  key: fs.readFileSync(config.tlsDir+'/tls.key'),
  cert: fs.readFileSync(config.tlsDir+'/tls.cert'),
  ca: fs.readFileSync(config.tlsDir+'/ca.pem')
}, function(req, res) {
  var form = new formidable.IncomingForm();
  form.uploadDir = config.uploadDir;
  form.parse(req, function(err, fields, files) {
    res.writeHead(200, {'content-type': 'text/plain'});
    res.end('upload received, thank you!\n');
  });
}).listen(config.port);

and allow people to post files to it with an html form like this:


<form action="https://example.com:3000/"
    enctype="multipart/form-data" method="post">
  <input type="file" name="datafile" size="40">
  <input type="submit" value="Send">
</form>

Indie Web

If you followed this episode, then you will have spent maybe 10 or 20 dollars, and probably about one working day working out how to fit all the pieces together and get it working, but you will have made a big leap in terms of your technological freedom: you are now a member of the "Indie Web" - the small guard of people who run their own webserver, independently from any big platforms. It's a bit like musicians who publish their own vinyls on small independent record labels. :) And it's definitely something you can be proud of. Now that it's up and running, add some nice pages to your website by simply uploading html files to your server.

From now on you will need your personal server in pretty much all coming episodes. For instance, next week we will explore a young but very powerful web technology, which will form an important basis for all communications between your personal server and the unhosted web apps you will be developing: WebSockets. And the week after that, we will connect your Indie Web server to Facebook and Twitter... exciting! :) Stay tuned, follow us, and spread the word: atom, mailing list, irc channel, twitter, facebook

Comments welcome!

Next: WebSockets