Dev server setup script

From LU
Jump to: navigation, search


To aid volunteers in contributing to their chapters' Web site and those of others, the LU Web Team has created a script that automates setup of a "development server," or "dev server" for short. A dev server is an instance of the Web site running within one of your personal computers. This instance is a working copy of our Git code repository that you can edit in order to experiment with features, bugfixes, and customizations to the site.

If you have a fast Internet connection, you might also consider using one of our virtual machine images.

What it does

The purpose of this script is to save you (the developer) time in getting your work started. This is what you have to do in order to set up your own dev server (and hence what the script does):

  1. Install software dependencies. These include database software (PostgreSQL and memcached), Python, many utilities, and many Python libraries.
  2. Check out a working copy of the Git repository containing the code.
  3. Add settings files specific to your installation. There are two files:
    • local_settings.py: Contains all Django settings that differ from the default configuration, except...
    • database_settings.py: Contains the DATABASES['default]['USER'] and DATABASES['default']['PASSWORD'] variables used by the database backend to connect to the database.
  4. (Optionally) Customize the site's media files. This is done using Dropbox. Each chapter can customize their Web site's media files, which include images, stylesheets (CSS), and uploaded content. To simplify distribution and backup of these customized files, we generally create a Dropbox for each chapter and place a 'media' directory in it. Each server of the site runs a Dropbox instance to synchronize the directory with the Dropbox server. Subdirectories within the 'media' directory are linked into the site's 'media' directory in order to override the default images and stylesheets.
  5. Set up a database. Commands are issued to PostgreSQL to create a user and a blank database, then to populate the database with the tables and data needed for the site (using Django's 'manage.py syncdb' and 'manage.py migrate' commands). You also have the option of loading from a database dump (.sql.gz file) if one is available.
  6. (Optionally) Set up Apache to host the site. You can skip this step and run the Django development server using 'manage.py runserver.'

How to use the script

  • Obtain a computer or virtual machine with a supported operating system. Currently supported operating systems are:
    • Ubuntu 12.04 64-bit
    • Ubuntu 12.10 64-bit
    • Ubuntu 13.04 64-bit
  • Obtain the latest version of the script here. You may need to make it executable (chmod +x dev_server_setup.sh) after saving it.
  • Run the script as follows:
 ./dev_server_setup.sh --all [absolute path to working directory] 
 # Example: ./dev_server_setup.sh --all ~/devsite

The path name for the last argument should not have a trailing slash or any dashes, otherwise the script will not work properly. You can use command line options to restrict the actions of the script if desired. Run './dev_server_setup.sh --help' to see those options.

  • The script will ask you many questions about the site configuration. This is to prevent you from having to dig through code files to type the site-specific information in all the right places. Please answer them.
  • After each stage of setup, the script will prompt you to "check the results." Please look at the output carefully. It may help to increase the size of your terminal's scrollback buffer to ensure you don't lose anything.
  • If you are setting up Dropbox, make sure you enter the appropriate Dropbox account information; each site has a Dropbox specifically for its media files. The script sets up its own instance of Dropbox to run concurrently with any other Dropboxes you may have on the computer. Sometimes a lot of stuff gets printed in your terminal during Dropbox setup and obscures the directions, which are to "type 'ok' and hit enter" once your machine has been linked to the account and it starts syncing. Dropbox setup is not necessary for generic feature development, e.g. GSoC work.
  • If anything goes wrong or you have a question, don't hesitate to e-mail us at web-team@learningu.org.

How to verify success / fix common errors

At this point, everything _should_ be set up properly. cd to <base-directory>/esp/esp, then try running "python manage.py runserver". If this works without any errors, go to localhost:8000. If the ESP homepage shows up, you're done! If any errors occur, let us know what they are so we can help you fix them. At this point, you should also be able to play around in the shell by running "python manage.py shell".

We have a separate page for common issues with dev servers: Dev server troubleshooting

Setting up encryption

If you are working with a chapter database, LU will require that you use a password-protected computer and additionally encrypt the database. There are a few ways to accomplish this:

  • If your dev server is a virtual machine, you can encrypt the virtual machine disk image file (e.g. VDI, VMDK) using the encryption capabilities of the host operating system.
  • You can create an encrypted partition on the dev server and place the database files on that partition.
  • You can encrypt the database files directly using a user-space encrypting filesystem like EncFS.

To use the latter approach, run the following commands:

sudo -s
BASEDIR=/var/lib/postgresql/9.1  # may depend on your PostgreSQL installation
DBNAME=devsite_django  # depends on what dirname you used when running the setup script
apt-get install encfs
sudo -u postgres mkdir $BASEDIR/encrypted $BASEDIR/encrypted.data
sudo -u postgres encfs $BASEDIR/encrypted $BASEDIR/encrypted.data
# [type 'p' for autoconfiguration]
# [enter a password]
sudo -u postgres mkdir $BASEDIR/encrypted/data
sudo -u postgres psql -c "CREATE TABLESPACE encrypted LOCATION '$BASEDIR/encrypted/data';"
sudo -u postgres psql -c "ALTER DATABASE $DBNAME SET TABLESPACE encrypted;"

Importing a database dump manually

To make a dump: run "sudo su postgres" then "pg_dump $DBNAME > /path/to/where/you/want/it".

In theory, you can import a database dump with ./dev_server_setup.sh --db or so, but that often fails. To do it manually, first get the database dump and gunzip it. Then run "sudo su postgres" and then "psql" to get an SQL prompt. From there, run (using the database username and password from database_settings.py, which ideally will be the same as those in the dump):

create role $USERNAME ;
alter role $USERNAME with password '$PASSWORD' ;
create database $DBNAME ;
grant all on database $DBNAME to $USERNAME ;

Then close the prompt with \q or Ctrl-D, and run "psql $DBNAME </path/to/dump.sql".