BuildAPI: 0.1

Getting StartedTo run BuildAPI locally, there were a few things that needed to be done, as outlined in the ReleaseEngineering/BuildAPI wiki (< Many thanks for that!!).

These were the steps:

1. Install MySQL and import database snapshots
2. Install Python and Python SetupTools (Easy_install)
3. Install Pylons (or use a virtual environment)
4. Install Google Python Visualizations Library
5. Install MySQL-Python (MySQLdb for Python)
6. Configure BuildAPI

 1. Installing MySQLInstalling MySQL is easy:

$ yum install mysql-server

Then create a new user and password for MySQL (optional), create the two databases (schedulerdb and statusdb) in MySQL and import the two snapshots into the appropriate databases. As noted in my previous post, I did run into problems getting the snapshots into MySQL, see here. However, that was resolved.


2.  Python and Python SetupTools
There were a few version of Python that I could have installed, however I chose to install version 2.6.6 as I had noted that the newest version 3.x was (maybe) not compatible with some of the components such as python mysqldb. I had used yum install to install it but it installed version 2.4 through yum so I downloaded the source and compiled it.
The python-setuptools package is necessary as it contains easy_install, which definitely lives up to its name and makes our lives soooo much easier (it resolves dependencies for python packages, just like yum)!

3. Install Pylons
What do you know, yum install pylons could work as well but instead I did easy_install pylons.


4. Install Google Python Visualizations Library
The visualizations library is used for, you guessed it, creating  graphical output such as graphs and charts for BuildAPI reports. After downloading the source code, I used python setup.py install to install it.

5. Install MySQL-Python (MySQLdb for Python)
This one took me a while to get installed and working. There were several dependencies that I needed to have installed before it could build, such as python-devel.
Smooth-sailing after that though.


6. BuildAPI
Once the BuildAPI source has been downloaded (buildapi source), I had to extract it to a directory called buildapi/ (by default it did not do this). Then run easy_install buildapi - it seems that easy_install goes into the folder and runs the python setup.py.

Once that is done, a default config.ini file was created (using paster make-config buildapi config.ini while in the buildapi directory) which would serve as the runtime configuration file. Changes needed to be made in here to allow BuildAPI to be accessible on the localhost and to be able to access the MySQL databases (SQLAlchemy is used with MySQL).

Thus, the following changes were made to the default config.ini file that was created using paster:
 

[server:main]

use = egg:Paste#http

host = 127.0.0.1

port = 5000

# SQLAlchemy database URL

sqlalchemy.scheduler_db.url = mysql://root:root@localhost/schedulerdb

sqlalchemy.status_db.url = mysql://root:root@localhost/statusdb

sqlalchemy.pool_recycle = 3600

That's it, that sets up BuildAPI to be accessible on http://127.0.0.1:5000/. Accessing it on the browser produces the following:

Each of the links work, as the database is set up as well. Here are a few example outputs:

MVC: Model–View–Controller

This is the concept that I need to understand in order to create the necessary code for the project. I noticed it is very similar to Perl's Templating Toolkit, where it allows you to separate the logic and view. 

From what I understand so far, the controller contains the logic, the model has the data and the view is basically the template that the end user will read the data from, as on a webpage for example.

Controller
When I am creating the scripts, I understand that the controller will call on the models or the views and return data to the model (please correct if this is wrong). It was also possible to use just a controller to show data on a webpage, however, it was not formatted. Using the command paster controller name created a default template controller that I could play around with.

Model

The model is where I can put the database connection and queries and store it into values that the template would then also use. I did read that for passing arguments from the model to the template, BuildAPI can use a function called "tmpl_context as c". Variables are called using the ${c.name} and this allows us to use the objects assigned as attributes of c anywhere else in the application - therefore, like global variables.

View
The model passes the data to the view/template which will display that information on a webpage with graphs and charts. These templates use a library set called Mako, which again, is similar to Perl's Templating Toolkit. It has many functions, expressions and tags that can be used. The template files are saved as name.mako

Here is a diagram of it (from wikipedia):

Well, this is how I understand the code will be created using MVC and its quite a lot to learn!

Testing 

One of the difficulties is knowing how all the components fit together, but after playing around with BuildAPI and taking a look at the structure of the pre-existing controllers, mako templates and model files, I have a fairly better understanding to progress into milestone 0.2.

I managed to create a few controllers myself, along with models and views using documentation on the internet (such as on PylonsHQ)

However, they were pretty simple and I need to learn more about the syntax and formats.
Example:

As an example, I was able to pull data (only used one column and a simple select query) and print it out on the screen using just a controller. I ran the command paster controller datatest, which created the template controller file and I modified that.
Note: To print to console, I had to edit the config.ini file and set

sqlalchemy.echo = True.

Simple Controller: datatest.py

from sqlalchemy import *

engine = create_engine('mysql://root:root@localhost/schedulerdb charset=utf8&use_unicode=0')
connection = engine.connect()

result = engine.execute("select name from schedulers")
for row in result:
        print "name:", row['name']
result.close()

This printed out all the names from the schedulers table from the schedulerdb database:

I believe I can get this to display on a webpage, but I need a little help. I can use a controller file, datatest.py, such as this:

import logging

from pylons import request, response, session, tmpl_context as c, url
from pylons.controllers.util import abort, redirect

from buildapi.lib.base import BaseController, render
from buildapi.model.getmydata import functionnname

log = logging.getLogger(__name__)

class DatatestController(BaseController):

    def index(self):
        # Return a rendered template
        c.message = GetMessage()
        return render('/displaydata.mako')

The return render will call on the displaydata.mako View/Template.
In the line

"from buildapi.model.getmydata import functionnname"

It will call the model getmydata.py (indicated by buildapi.model.getmydata - i.e. directory/directory/file) and in this file is where I could define the functionname.
For example, if the function name is GetMessage, then it would be defined as:

def GetMessage():
    return "Gimme my data fool!"

In the template, I simply use ${c.message}to get the data. If it is defined as above, then in the webpage (view) that is displayed, it will replace ${c.message} with "Gimme my data fool!".
Note: as shown in the controller file, datatest.py, c.message was defined as:

c.message = GetMessage()

Expectations

• I would like to be able to print the data out to a webpage but will need to understand MVC and pylons syntax more. I will use the resources as listed on the BuildAPI wiki to help me with this
• Once that is done, then I will try to put that data into a graph or chart
• Next would be to pull the correct data and generate graphs based on the project objectives
• And finally, get it all documented!

Quick Note about Python Virtual Environment

A python virtual environment was highly recommended by  Release Engineering, and therefore I also set up one for testing BuildAPI code before I brought it over to my actual installation on Fedora. It was extremely easy to set up, and literally took a few minutes to get it running.
Using easy_install, you can simply use easy_install virtualenv and it will install the virtualenv script to your path. Then you just need to create a directory, run the script wit the directory as your argument and it will set up a sandbox environment. Inside the sandbox/bin folder is an activate script, which as the name implies, activates the virtual environment.

Here are the commands if you are interested:
$ easy_install virtualenv

$ mkdir ~/venv

$ virtualenv ~/venv/sandbox

$ source ~/venv/sandbox/bin/activate

(sandbox)$ easy_install [packagename]

Once you're in the sandbox, you can install whatever else you need to install 

I kept detailed instructions on how to do each step, and I will leave that for the final documentation.