Deployment Environment

Installing a Global Virtual Environment

It is best practice to install PubliForge into a “virtual” Python environment in order to obtain isolation from any “system” packages you’ve got installed in your Python version. This can be done by using the virtualenv package. Using a virtualenv will also prevent PubliForge from globally installing versions of packages that are not compatible with your system Python.

This documentation assumes that Python and virtualenv are already installed and working in your system.

Install a global virtual environment for all your instances of PubliForge. To do so, invoke the following:

$ sudo virtualenv /usr/local/virtualenv
$ sudo chown $USER -R /usr/local/virtualenv
$ source /usr/local/virtualenv/bin/activate
(virtualenv)$ pip install -U pip setuptools

Adding a User

Create a user for your instance so you can control use of resources (Cf. ulimit, quota…). In this documentation, we suppose the name of this user is pfinstance.

$ sudo adduser pfinstance

Make sure that on your system an UTF-8 locale is generated. To do so, type:

$ sudo dpkg-reconfigure locales

Installing Required Packages

Before installing PubliForge, you must have on your system the following packages:

$ sudo aptitude install python-dev libxml2-dev libxslt1-dev librsync-dev libpq-dev libaprutil1-dev libsvn-dev

If you plan to use a MySQL Database:

$ sudo aptitude install libmysqlclient-dev

For LePrisme module, you also need the following packages:

$ sudo aptitude install yui-compressor
$ sudo aptitude install imagemagick optipng advancecomp jpegoptim
$ sudo aptitude install ffmpeg

If you plan to make ePub, you will need a Java Runtime Environment to run ePubCheck.

If you plan to make PDF or mathematical formulas via LaTeX, you will need:

$ sudo aptitude install dvipng
$ sudo aptitude install texlive texlive-luatex texlive-xetex texlive-latex-extra texlive-fonts-extra
$ sudo aptitude install texlive-lang-french

In the last command, change french with your language.

Setting up the Database

You need to create the database that matches your future configuration.

For instance, if you decide to use PostgreSQL (the recommended database), invoke something like:

$ sudo aptitude install postgresql
$ sudo su - postgres -c "createuser --no-superuser --pwprompt publiforge"
$ sudo su - postgres -c "createdb -T template0 -E UTF-8 -O publiforge PFInstance"

The default password is sesame.

If you decide to use MySQL, install the follwing Python package:

$ source /usr/local/virtualenv/bin/activate
(virtualenv)$ pip install MySQL-python

Then, install MySQL:

$ sudo aptitude install mysql-server
$ mysql -u root -p mysql
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,INDEX,CREATE,DROP ON *.* TO publiforge@localhost IDENTIFIED BY 'sesame';
mysql> CREATE DATABASE PubliForge;
mysql> QUIT;

Installing PubliForge

Now, it is time to check out sources and install them in your virtual environment:

$ source /usr/local/virtualenv/bin/activate
(virtualenv)$ pip install Mercurial
(virtualenv)$ hg clone https://hg.publiforge.org/PubliForge
(virtualenv)$ cd PubliForge
(virtualenv)$ pip install .

You can also use pip install -e . instead of pip install . to keep the link between your installation and PubliForge sources. In that case, you can create the cached byte-code files:

(virtualenv)$ python -m compileall publiforge/

Setting Up PubliForge Environment

Within pfinstance home directory, make the following structure:

$ su pfinstance
$ cd
$ mkdir PubliForge
$ cd PubliForge
$ mkdir Cache Storages Builds Buildspaces

And create a pfinstance.ini (or a production.ini) file like the following:

# =============================================================================
[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = %(http_port)s
#numthreads = 10
#timeout = 180
#request_queue_size = 200

# =============================================================================
[app:main]
use = egg:PubliForge

# ------ Instance defintion
uid = pfinstance
encryption = secretkey
#maintenance = true

# ------ Internationalization
languages = fr, en
#pyramid.default_locale_name = en
#translation_dirs =

# ------ Debug mode off
pyramid.reload_templates = false
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.debug_templates = false

# ------------------------------ Front

# ------ Skin
skin.name = pfinstance
skin.title = PubliForge – PFInstance
#skin.static.name =
#skin.static.path =
#skin.favicon =
#skin.css.base =
#skin.css.login =
#skin.css.custom =
#skin.template.base =
#skin.template.login =

# ------ Temporary directory
temporary_dir = %(here)s/Cache/Tmp

# ------ Beaker cache
#cache.type = file
#cache.data_dir = %(here)s/Cache/BeakerCache/Data
#cache.lock_dir = %(here)s/Cache/BeakerCache/Lock

# ------ Beaker session
#session.type = file
#session.data_dir = %(here)s/Cache/BeakerSessions/Data
#session.lock_dir = %(here)s/Cache/BeakerSessions/Lock
session.key = PF_SESSION

# ------ SQLAlchemy
# ... PostgreSQL
sqlalchemy.url = postgresql://publiforge:sesame@localhost/PFInstance
#sqlalchemy.pool_size = 5
#sqlalchemy.max_overflow = 10
#sqlalchemy.pool_timeout = 30
# ... MySQL
#sqlalchemy.url = mysql://publiforge:sesame@localhost/PFInstance?charset=utf8
#sqlalchemy.pool_recycle = 3600

# ------ Authentication
#auth.cookie = PF_AUTH
#auth.remember = 5184000

# ------ Page refresh
#refresh.short = 2
#refresh.long = 5

# ------ Storages
storage.root = %(here)s/Storages
storage.index = %(here)s/StoragesIndex
storage.vcs = none, local, hg, hgsvn
#storage.report_ttl = 120
#storage.cache = 3600

# ------ File openers
opener.roots = %(here)s/Openers
opener.list = Publi*, Ini*
opener.cache = %(here)s/Cache/Openers

# ------ Available agents
#agent.refresh = 0
#agent.synchronize = 1
agent.0.weight = 1
agent.0.password = sesame

# ------------------------------ Agent

# ------ Buildspaces
buildspace.root = %(here)s/Buildspaces
#buildspace.ttl = 2678400

# ------ Processors
processor.roots = %(here)s/Processors
processor.list = Publi*, PackFilling, Parallel, CharCount, DTBookValid

# ------ Builds
#build.reset = false
build.root = %(here)s/Builds
#build.ttl = 1800
#build.result_ttl = 604800
#build.concurrent = 3

# ------ Authorized fronts
front.0.uid = pfinstance
front.0.password = sesame

# =============================================================================
[Populate]
# ------ Administrator
admin.login = admin
admin.password = adminpwd
admin.name = Administrateur
admin.email = admin@prismallia.fr

# =============================================================================
# ====== Logging configuration
[loggers]
keys = root, publiforge, sqlalchemy, activity

[handlers]
keys = console, activity

[formatters]
keys = generic, filelog

# ====== Loggers
[logger_root]
level = WARN
handlers = console

[logger_publiforge]
level = WARN
handlers =
qualname = publiforge

[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine

[logger_activity]
level = INFO
handlers = activity
qualname = activity
propagate = 0

# ====== Handlers
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[handler_activity]
class=handlers.TimedRotatingFileHandler
args=('%(here)s/Log/publiforge.log', 'w6', 1, 52, 'utf-8', True)
level=NOTSET
formatter=filelog

# ====== Formatters
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s

[formatter_filelog]
format = %(asctime)s %(message)s

You can read Customization with INI File for a precise customization.

Populating the Database and the Storages

Use the pfpopulate console script to populate your database and your storages. To do so, type the following command:

$ source /usr/local/virtualenv/bin/activate
(virtualenv)$ pfpopulate pfinstance.ini

Activating Job Scheduler

A background job is a processing which is periodically run by a scheduler. If you plan to use this functionality, you have to configure the CRON table of the PubliForge user. To do so, execute the following command:

$ su pfinstance
pfinstance$ crontab -e

And enter a configuration such as:

*/6 * * * *  /usr/local/virtualenv/bin/pfscheduler ~/PubliForge/pfinstance.ini

Now, every 6 minutes, the PubliForge scheduler will run the pending jobs.

Deploying

You can now deploy your instance with Apache (see Deployment with Apache and mod_wsgi) or Nginx (see Deployment with Nginx and supervisord).