Maxwell Swadling

How to Write Documents

2013-03-26T00:00:00-07:00

This is my "getting started" with writing documents guide.

Write Document

First step: write the document in plain text (I use markdown). Plain text is easy to diff, put into source control, collaborate on, plus there are some great editors out there with tons of tools.

Makefile

Make a Makefile. Makefiles define a process to convert forms of data, such as a plain text document into a type setted document, plus are platform agnostic. A simple example would be:

%.pdf : %.tex
  pdflatex $<
  pdflatex $<
  pdflatex $<

%.tex.in : %.md
  multimarkdown -t latex -o $@ $<

%.tex : %.tex.in
  m4 template.m4 > $@

default: report.pdf

template.md

Make a template.m4 to template the outputted latex into a template. Remember your plain text document only specifies the document, not any formatting rules. So you can define some macros such as:

divert(-1)
dnl Report name
define(`REPORT_NAME', `Assignment 1')
define(`chapter', `section*')
divert(0)

The REPORT_NAME macro is just a constant I can use in the markdown. The chapter lets me use ## Title in markdown and it will convert it to a unnumbered section.

I find it useful to shell out to get the git commit I am on to embed into the report.

define(`GIT_COMMIT', `esyscmd(git rev-parse HEAD)')

You can make more interesting macros like for including figures on the page or forcing page breaks from markdown.

define(`FIGURE', `\begin{figure}[]
  \centering
  $1
  \caption{$2}
  \label{fig:$3}
  \end{figure}')
define(`PAGEBREAK', `\pagebreak')

After you have defined the macros you want and divert(0), you should then include a template to wrap your latex. To insert your document, include sinclude() in your template (in the body) with the filename you want to include. You could do this and the figure macro with the latex processor, but I prefer to use m4.

\documentclass[12pt]{article}

\usepackage{fancyhdr}
\usepackage[usenames,dvipsnames]{color}
\usepackage{graphicx}
\usepackage{listings}

\usepackage[pdftex,
        pdfauthor={AUTHOR_NAME},
        pdftitle={REPORT_NAME}]{hyperref}

\topmargin=-0.4in
\evensidemargin=0in
\oddsidemargin=0in
\textwidth=6.0in
\textheight=9.0in
\headsep=0.25in
\setlength{\parindent}{1cm}
\linespread{1.1}

% header / footer
\lhead{}
\chead{REPORT_NAME}
\rhead{}
\lfoot{AUTHOR_NAME}
\cfoot{}
\rfoot{Page \thepage}
\renewcommand\headrulewidth{0.4pt}
\renewcommand\footrulewidth{0.4pt}
\setlength\parindent{0pt}

\pagestyle{fancy}

\begin{document}
\maketitle

sinclude(`report.tex.in')

\end{document}

Now you have a nice looking pdf!

File Formats

2013-01-19T00:00:00-08:00

Choosing to use plain text file formats simplifies a lot of problems. You can use pretty much any text editor. You can sync via rsync, Dropbox, etc. If you need revision control, you can use Git on the files. Since they are just plain text, you can diff, merge, etc.

Useful file formats:

Documents (.md)
Typesetting (.latex)
Spreadsheets / Data (.yaml / .json with Ruby / Haskell)
Presentations (.md with Showoff)

If you choose to use Markdown, there are plenty of tools to export to formats such as LaTeX or HTML, however, it is trivial to define your own format, like my Creole export.

For Spreadsheets I have a library called Sheet.hs.

DegreeSolver.hs

2012-12-13T00:00:00-08:00

Download DegreeSolver.hs • GitHub Source

Today I wrote a program to solve which parts of your degree have not yet been completed. It's a beefed up replacement to an old spreadsheet I have been using for years.

Since everyone is special, I recommend you read your program outline and create an appropriate file for yourself.

Firstly, get the program off GitHub. It's written in Haskell so you're going to need to install ghc or ask me for a binary.

To run it:

$ runhaskell DegreeSolver.hs program.data completed.data

program.data

This defines a program. A program is made of requirements. For example, a program that:

must completed COMP0001
any 12 units of electives
either MATH101 or MATH102

Can be defined as:

[Req 6 "COMP101", Either 6 "MATH101" "MATH102", Any 12 "ELECTIVE"]

completed.data

This defines the courses completed so far.

C is for Core courses (Req or Either)
X is for Elective courses (Any)

To say you have done MATH102 and 6 units of electives use:

[C "MATH102", X "FUN101" "ELECTIVE"]

Output

The program will tell you:

Which courses you haven't completed
How many of each elective you have left
If you have done a course that doesn't count

Example:

Incomplete Core Courses:
[Req 6 "COMP101"]

Units Remaining in Electives:
ELECTIVE:   6

Completed Courses:
(C "MATH102",Just (Either 6 "MATH101" "MATH102"))
(X "FUN101" "ELECTIVE",Just (Any 12 "ELECTIVE"))

What this doesn't do: Program changes, exceptions, waivers, etc.

ditch

2012-09-30T15:04:00-07:00

Evernote updated Skitch a couple of days ago and removed WebDAV. They were nice enough to provide a download of the old version from their site.

I made a Ruby web app you can run for free on Heroku. Just plug in a Dropbox API key, and you can point Skitch's WebDAV at it and it will store the images into your dropbox!

It uses the Dropbox App Folders, so when you give it permission to use your Dropbox, it can only see and use the /Apps/ditch directory. More apps should do that!

There is a problem with Skitch's WebDAV and the clipboard. It verifies the URL exists after uploading. Unfortunately this means you can't return Dropbox link since it is just a hash / number. Also if you return a header location redirect, it will not accept it. So instead it copies to the clipboard a link to a page that redirects the user to the right Dropbox url.

Event-B Conversion

2012-08-12T14:18:00-07:00

To show how to do conversion from Event-B to code, I have made a simple machine to show how every part of Event-B is converted. For the example I have made a Microblogging service.

Event-B Component

User Machine

For the Event-B model we first make a User machine.

For the invariants; We make a variable called users and give it the invariants: users ⊆ USERS. USERS is defined in a Users context as a SET.

users is a subset of USERS. It contains all the currently registered users of the system.

Next we make an event. create_user takes any USERS who is not yet in users and adds them to users. The 2 guards are the user is in USERS and the user isn't yet in users.

Status

Next we refine in the Status machine. We add a new variable called statuses to represent all the status updates made by users. We make a context called Statuses that defines the set STATUSES. statuses will have an invariant that enforces all statuses are in STATUSES such as statuses ⊆ STATUSES.

Also we need to map a status to a user, so we make a mapping called owners with the invariant owners ∈ statuses → users.

Now we can make an event called post_status. If we make it like this:

post_status:
        ANY
            status
        WHERE
            grd1:   status ∈ STATUSES not theorem
            grd2:   status ∉ statuses not theorem
        THEN
            act1:   statuses ≔ {status} ∪ statuses
        END

We get a failed proof obligation that owners ∈ statuses → users doesn't hold! Which is correct. So we need to add the action for owners.

post_status:
            ANY
                status
                user
            WHERE
                grd1:   status ∈ STATUSES not theorem
                grd2:   status ∉ statuses not theorem
                grd3:   user ∈ users not theorem
            THEN
                act1:   statuses ≔ {status} ∪ statuses
                act2:   owners(status) ≔ user
            END

Now we can be sure that all status updates will have an owner.

Conversion

User

If we did have an event like delete_user that required a user ∈ users then it would be an instance method.

First we take the invariants and map them to validations. User only has one, users ⊆ USERS. This becomes an ID to uniquely identify a user.

Next we implement create_user. Since create_user doesn't have user ∈ users, this is a class method. An instance method requires user ∈ users, since user will then be bound to self or this.

It does take an id, but we are going to assign ids ourself, so create_user just becomes the Active Record Pattern's .create class method.

The id must be present to exist, so we insert a presence validation. Also the id is used to identify a unique instance, so it also requires a uniqueness validation. However, most implementations of the Active Record Pattern do ids like this automatically, so a correct implementation of user could be:

class User < ActiveRecord::Base
  # ids are done by the super class
end

Status

Each refinement is a new model, so here we create a new model for Status. It too has an ID, statuses ⊆ STATUSES. However, it has one extra invariant. Since each invariant maps to a validation, we must map this invariant too.

owners ∈ statuses → users

This means, given a status, you can determine the owner of it. This means we have a belongs_to relation. To match this, the User model needs a has_many :statuses relation as well.

Also since it is a total function, we must validate the user_id is present.

class Status < ActiveRecord::Base
  belongs_to :user

  validates :user_id, :presence => true
end

We could change the name from user_id to owner_id if we wanted to match the Event-B model's name for this function.

Looking at the guards for post_status this could be a class method in Status. This is exactly what it is by default. However, we should notice the user ∈ users guard. This means we could instead make post_status an instance method on a User! Then self would be bound to the user making the post and the only argument would be the status to post.

Conclusion

Each refinement becomes a Model
Each variable in the refinement becomes a member variable
Each invariant became a Model Validation
Each event becomes a method
The machine states become state machines inside the Model

You can download the Event-B Model.

Note on Authorisation

An Event-B model represents an Active Record Pattern system. In MVC this is the Models.
Authorisation in MVC is part of the controller.
Therefore you can not model with Event-B authorisation since it is outside what Event-B can represent.

Raspberry Pi + Ruby

2012-08-06T22:41:00-07:00

Today I got a Raspberry Pi. I currently run a laptop that has a couple of ruby apps. I wanted to move these apps to something less power intensive and hot.

Before I could use it, I needed 2 things:

SD Card 8GB ($16)
Micro USB Charger ($20) (1000mA) from a phone shop

Add the postage of it, and it bumps the price from $35 to $80 computer.

First download the debian image.

For imaging the SD card, check out this guide. For Mac, df -h to list drives, sudo diskutil unmount /dev/diskXsN to unmount and sudo dd bs=1m if=something.img of=/dev/rdiskX, eject with sudo diskutil eject /dev/diskX.

Once you start it up the first time, run sudo raspi-config and resize partition to fill SD card. Then restart it. Also remember to sudo apt-get update && sudo apt-get upgrade.

To make SSH-ing in easier I made the following ~/.ssh/config

Host pi
  HostName pi.maxpow4h.com
  User pi

Then I ran ssh-copy-id pi, so I can just use ssh pi.

Now if we want to do anything of interesting we need some stuff:

sudo apt-get install git curl zlib1g-dev # for RVM
sudo apt-get install openssl libreadline6-dev git-core zlib1g libssl-dev # for ruby's ssl
sudo apt-get install libyaml-dev libsqlite3-dev sqlite3 # for ruby's yaml and sqlite3
sudo apt-get install libxml2-dev libxslt-dev # for nokogiri
sudo dpkg-reconfigure tzdata # set your time zone

RVM time! curl -L https://get.rvm.io | bash -s stable --ruby. Compiling ruby takes a long time...

Now we can run our apps!

mkdir app_name
cd app_name
git init
git config receive.denyCurrentBranch ignore

vi .git/hooks/post-receive
# insert the following into the post-receive hook
# cd ..
# GIT_DIR='.git'
# git checkout -f

chmod +x .git/hooks/post-receive

Now on your computer lets add the remote:

git remote add raspberry pi@192.168.0.123:app_name/.git
git push -u raspberry master

Now it will keep the working dir up to date when you push.

Back on the server. If you are running a service and you want it to be monitored, the easiest way is to add a Procfile with the commands to run and use bluepill to monitor it.

bundle install

# Now for the Procfile
gem install bluepill foreman

foreman export bluepill . -u pi

rvmsudo bluepill load app.pill
rvmsudo bluepill status

Now you can checkin on how everything is going with rvmsudo bluepill status. Check the logs with rvmsudo bluepill log <process_or_group_name>. Start / stop with rvmsudo bluepill <start|stop|restart|unmonitor> <process_or_group_name>. Also you probably want to add rvmsudo bluepill restart app to the post-receive hook so the app is restarted each push.

From here, you probably want to install transmission and enable the web interface. Then install nginx and reverse proxy your web app.

location / {
  proxy_pass        http://localhost:9292;
  proxy_set_header  X-Real-IP  $remote_addr;
}

An alternative to this is using the ruby Puma gem for the web server. I've found it works well with video streaming.

Also check out elcuervo/airplay for streaming to AppleTV.

Also you might want to proxy and rewrite transmission something like this.

location = /transmission {
   rewrite /transmission/(.*) /$1 break;
   proxy_set_header   Host        $http_host;
   proxy_set_header   X-Real-IP   $remote_addr;
   proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
   proxy_pass         http://127.0.0.1:9091/transmission/web/;
   proxy_redirect     off;
}

Also adafruit make a really neat case. It is worth checking out.

md2cre.rb

2012-07-22T17:21:00-07:00

I write everything in Markdown, but a piece of software I have to use requires Creole. So, I made this script that converts Markdown to Creole.

Event-B and You

2012-07-16T00:01:00-07:00

Translating Event-B (or any state based modelling language) to an implementation can be tricky. Here is the process I used to do it. The first step is to identify what you are implementing in. This is the Active Record Pattern. It exists in many different programming paradigms, such as:

Callbacks: Event-Driven Programming
MVC: Object Orientated Programming
Monads: Functional Programming

The Process

Note: "Model" refers to the Active Record Pattern idea of a model, not the Event-B model.

Each refinement because a Model.
Each variable in the refinement because a member variable.
Each invariant became a Model Validation.
Each event becomes a method.
The machine states become state machines inside the Model.

Then, no data could be modified unless all the invariants you specified in Event-B held true for your Models.

The next / most important part of the Event-B Model, are state machines. This requires you to define states (which is an enumeration). Then each invariant on those states you specify as state transitions with guards (in your model). Then you can define methods to require certain states. This then results in events been protected by the state of the machine, and only certain state transitions been allowed.

This process will map your complete Event-B model into any language's Active Record Model. Then on top of that you can use any design pattern like MVC, Model2, etc.

Guardfile

2012-04-28T21:54:00-07:00

Guard is a gem that lets you setup things to happen when stuff is changed. I use it for previewing a PDF while I write LaTeX.

To set it up, install the gems guard and guard-shell. Then make a Guardfile with:

guard :shell, :all_on_start => true do
  watch /cv.tex/ do |m|
    3.times {system "pdflatex cv.tex"}
  end
end

or something more fancy

guard :shell, :all_on_start => true do
  watch /^([^\/]*)\.tex/ do |m|
    3.times {system "pdflatex -shell-escape #{m[0]}"}
  end
end

Now run guard! Open cv.pdf in your pdf viewer and now every time you change cv.tex, the pdf will be regenerated automagically!

Graphviz Positioning

2012-03-29T17:02:00-07:00

Positioning elements in Graphviz is a little bit tricky sometimes. Here is how I do it.

digraph finite_state_machine {
  # Settings
  rankdir=LR;
  size=8

  ## Nodes
  node [shape = doublecircle] waiting_for_input finished;
  node [shape = circle] find_path_back check_found;

  # Lines
  waiting_for_input -> find_neighbours [  label="user input"];
  find_neighbours -> check_found [label="goal not found"];
  check_found -> find_neighbours [label="not found"];
  check_found -> find_path_back [label="goal found"];
  find_path_back -> finished [label="display result"];

  # Layout
  { rank=same; waiting_for_input waiting_for_input }
  { rank=same; find_path_back check_found}

}

Then to generate it use

dot -Tpdf diagram.gz > diagram.pdf

If you don't use dot it may ignore your positioning. E.g. the Graphviz GUI, neato, etc.

OS161 on Homebrew

2012-03-01T16:06:00-08:00

I built OS161 for Homebrew. If you need it too, run:

brew tap maxpow4h/os161
brew install sys161

The repo is available on maxpow4h/os161.

TextMate 2 Properties

2012-02-21T16:28:00-08:00

The TextMate 2 .tm_properties global file I use:

tabSize = 2
softTabs = true
softWrap = true

[ Makefile ]
softTabs = false
tabSize = 4

excludeDirectories doesn't remove the results from finding. So I use exclude instead.

For Jekyll projects:

exclude = "{_deploy,public,source/stylesheets}"

For Ruby projects:

exclude = "{tmp,log}"

For C things:

excludeFiles   = "*.{so,o}"

K17 iOS App

2012-02-04T21:14:00-08:00

This app is something I have wanted to make since first year. The cse_pp script provides an API for the iOS app to search for information with.

insta-importo.rb

2012-01-26T23:45:00-08:00

I wrote a script to import the articles from The Daily WTF that I haven't read yet into Instapaper.

It is also handy for other blogs.

Git at University

2011-12-20T00:00:00-08:00

Remote

If you want to use a server's plain old filesystem to host your git repo, like at university, it's pretty straight forward.

# on the server
mkdir ~/ass1; cd ~/ass1
git init
git config receive.denycurrentbranch ignore

# on your computer in the git repo
git remote add cse USERNAME@cse.unsw.edu.au:ass1/.git
git push -u cse master

Hooks

One useful thing about using Git Hooks is you can submit your assignment and see the result of the dry run, just by using git push.

To set this up on the server go to the repo and create a .git/hooks/post-receive file. This is the script that gets executed after a push. It should contain:

#!/bin/bash

# This checks out the push into the working dir
GIT_WORK_TREE=`pwd`/.. git checkout -f
echo "Giving..."
pushd ..

# put your give here...
# give cs1234 ass1 file.c 

popd

Then run chmox +x hooks/post-receive to make it executable. Now every time you push it will update the working tree.

Note: This is very sensitive to git versions. git version 1.7.x.x where x differs may not be compatible. For Ubuntu add the ppa for the latest version.

Also Note: You can't use #!/bin/sh if you expect pushd to work.

SVN

Some courses use SVN, to fix that:

git svn clone svn+ssh://maxs@cse.unsw.edu.au/home/maxs/cs3231/repo
# work normally in git
# make commits, etc.
# then when you want to commit to svn, do
git svn fetch # you don't want to do a merge in svn...
git svn dcommit
# Also on the server, you need to go 'up' a few levels of commits
# so use `svn up -r1234`

Ruby Without Bundler In Production

2011-12-16T13:15:00-08:00

Sometimes Bundler isn't available in the environment you want to run a ruby app on. Oh noes. This is how I get around that problem.

Download the .gem files to my local machine
Deploy to production
Install gems
When starting, set the GEM_HOME and GEM_PATH

Here are the Rake tasks I use:

Graphviz Class Diagram

2011-09-21T00:00:00-07:00

An example of a class diagram in Graphviz.

LaTeX Template Tools for SENG

2011-08-24T00:00:00-07:00

The tools my group used last year for SENG. Used for templating requirements into LaTeX tables.

Email Stats

2011-07-17T00:00:00-07:00

A small ruby script that downloads your IMAP email headers into a local database. You can then query it for statistics, such as ranking where most your emails come from.

Task Warrior .taskrc

2011-07-11T00:00:00-07:00

Task Warrior is great. Here are 2 things I added to my .taskrc to make it better.

Backup

Have task warrior use a folder inside Dropbox. Now its backed up!

# Files
data.location=/Users/maxs/Dropbox/Documents/task

Date Format

Set it to use Australian date formats.

dateformat=d/m/y
dateformat.report=d/m/y
dateformat.annotation=d/m/y

OS X Failed Backup Alert

2011-07-10T00:00:00-07:00

A script to email you when a Time Machine backup has failed in the last 48 hours. You need to fill in the email destination and smtp settings. Works with system ruby, just install the pony gem.

A DVD of Prime Numbers

2011-06-29T00:00:00-07:00

My friend once downloaded a DVD of prime numbers. He wanted to match them up to other things. At the time, Internet was slow and a DVD took a while to download. I thought, "It must be quicker to just generate them than download them". However, I had no idea how to do it at the time. Now I do, and it turns out, yes, it is quicker.

bookmarks.rb

2011-06-28T00:00:00-07:00

A simple script I made that tells you where most of your bookmarks in Chrome are from and if you have duplicates. You need to export your bookmarks from Chrome to use it.