Making LaTeX easier: Maintaining your own local repository

Before we start I want to make clear by “your own local repository” I am not suggesting installing all packages manually. That would make things harder, not easier! Instead I am suggesting a local repository for your own files. Do you have a .bib file you use for most of your work? A version you are forever copying into new folders and modifying (or perhaps it is output from a reference manager such as Mendeley). Not to mention custom .bib and .cls files provided by journals, and other .sty files which are not packaged and available via MiKTeX or TeXlive.

The bad way to handle this is to keep copying these files to new folders each time you need them. The difficult method is to start modifying the folders maintained by TeXlive and MiKTeX. The best method is to make a folder for these files on your computer and tell TeXlive or MiKTeX to always check this folder. This way you can put it in a Dropbox folder (or other cloud synchronised folder) and have these files update on any machine you use.

So how do we do this? First we need to make a directory, called whatever you want, I will call this ‘MyTeX’. In side this we need a specific file structure. Make the following directories:
MyTeX/bibtex
MyTeX/bibtex/bib
MyTeX/bibtex/bib/misc
MyTeX/bibtex/bst
MyTeX/bibtex/bst/misc
MyTeX/tex/latex
MyTeX/tex/latex/misc

Now put any .bib files in the MyTeX/bibtex/bib directory, any .bst files in the MyTeX/bibtex/bst directory, and any .sty or .cls files in MyTeX/tex/latex (For packages which have a whole directory of files you can just copy the directory into MyTeX/tex/latex).

Now we have our local repository we need to tell LaTeX about it. This needs to be done differently in Windows and Linux

In Windows

It is very easy to do this in Windows with MiKTeX. Simply go Start -> All Programs -> MiKTeX -> Maintainance (Admin) -> Settings (Admin). Which brings up this window (switch to the Roots tab):

MiKTeX1

Click “Add…” and navigate to the MyTeX folder (or whatever you called it) and add it.  It should now be listed in that big white box. Switch to the General tab:

MiKTeX2

And here we want to refresh the file name database (I think that is what it stands for), so we click “Refresh FNDB”. Once that is finished click OK.

That was it! You can check by putting a .bib in the bib directory (or a .bst in the bst directory, or a .sty in the latex directory) and check that when you run LaTeX in a nother directory it can find them.

Linux

I have to say that while in I think Linux is much better than Windows on most things. I am surprised to have to say that its support for TeX repositories (at least on Ubuntu) is shockingly bad. On an operating system where from command line you can automatically install and update most software it is a bit pathetic that the version of TeXlive available on Ubuntu does not have the ability to install missing packages on the fly (especially when Windows can). Doing a TeXlive full install get’s around this, but I don’t see why I should need to waste so much space on packages I don’t need just so I have access to ones which I do which aren’t in the basic TeXlive.

I am sorry also to say that setting up your own TeX repository is also a lot less hassle on Windows. But once it is done there is no more hassle. So rant over, let’s do this!

  •  First we need to locate some TeX config files. Open the terminal and run:
    kpsewhich texmf.cnf
    to locate the texmf config file. Open this in gedit (no need to open with sudo) by running:
    gedit %FILENAME% &
    (the & is optional but it stops the terminal waiting for gedit to close)
  • The file ask you not to edit it. But tells you the directory where it is generated from.
    go to this directory by running:
    cd %path_in_file%
  • Let’s look at the files in the directory by running:
    ls
  • You could edit one of these files or (should end with .cnf), but let’s create our own with the touch command. I think (but I am not sure) they are included alphabetically and I have had some problems using an arbitrary name, so I find it is best to put it first by having a file name which begins with 00. For example: 00JulianRepos.cnf
    So you should run:
    sudo touch %name_you_chose%.cnf
    Then run (to edit file):
    gksu gedit %name_you_chose%.cnf
    gksu is like sudo but has a GUI window asking for permission before the program opens and it sets the permissions correctly for the GUI. There is no & on the line so we can’t accidentally forget to save and close the file.
  • Add the following line to the file:
    TEXMFHOME=%FULL_PATH_TO_MyTeX_DIRECTORY_YOU_MADE%
  • Save file and exit gedit.
  • We now need to update texmf.cnf by running:
    sudo update-texmf
  • Now we update the LaTeX’s map of where things are by running:
    sudo updmap
  • Now let’s check it worked. Run:
    kpsepath tex | tr : '\n'
    You should see a list of directories. Your local repository followed by: /tex///
    should be in this list. This means it is checking the tex folder of your archive
    and all its subdirectories for tex files (and .sty and .cls files)
  • As a final check we can look for a file in the repository. Pick a file which is in one of the sub folders of the repository and add the filename (with no extension)  into the following command:
    kpsewhich %FILENAME%
    If everything is working it should give you the full path to that file when you run the command.

That was it! You are now done.

Setting up word count for LaTeX

Occasionally it can be useful to have a word count for a document I am writing. Being a nerd who hates word processors and tends to use a lot of maths in most things I write, I use LaTeX.

So how can you get a reliable word count in LaTeX? There is a brilliant program called TeXcount which word counts a LaTeX file. It has many options and can output to the terminal or give a detailed summary in HTML. TeXcount is included with many LaTeX distributions, or can be downloaded from the link above. No need to install, just unzip the files. But getting it to run nicely can take a couple of tricks.

Perl

First, TeXcount is written in perl so you will need perl to run it. If you are on a unix system you should have perl already. On windows you will need to install perl. To install oerl simply run the strawberry perl installer.

Options for a nice display

TeXcount has a great number of options. You can explore these by reading the PDF the comes with TeXcount. For me I run with the three following options:

  • -html     – This makes the output HTML code which when displayed in a browser give a nice overview of the number of words, words in headings words in caption, equations etc.
  • -inc    – This tells it to also word count any extra .tex files included in the .tex file you ran.
  • -incbib  – This tells TeXcount to also word count the bibliography, you may or may not want this.

Actually running TeXcount

Of course you can just run TeXcount from the terminal with

texcount.pl -html -inc -incbib myfile.html

or if texcount is not in the PATH

path/to/texcount/texcount.pl -html -inc -incbib myfile.html

(using \ not / if you are in Windows). This is not to convenient because it the puts all the HTML into the terminal. You can of course use > to redirect the output into a file. But opening the terminal and finding the directory and writing (or reverse searching) the commands is tedious. Instead one can write a simple 3-4 line script to run TeXcount and displays the output in a browser.

UNIX – save this code as “RunTexcount.sh” in the same folder as TeXcount (for KDE you can change “gnome-open” to “kde-open”, or mac simply “open”):

#!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
"$DIR"/texcount.pl -html -inc -incbib "$1".tex > "$1"_count.html
gnome-open "$1"_count.html

Windows- save this code as “RunTexcount.bat” in the same folder as TeXcount:

@echo off
%~dp0\texcount.pl -html -inc -incbib "%1".tex > "%1"_count.html
"%1"_count.html

Now any LaTeX IDE worth using has a way of setting up custom executables. (In Kile go to Settings>Configure Kile>Build>New, in WinShell go to Options>User programs.)
Here you can simply set the .sh or .bat file as the executable. For the options (cmd line) you need the name of the .tex file WITHOUT the file extension. This can usually be done using '%s' or "%s". The easiest thing to do is look to see how BibTeX is set to run and use the same commands.

Now, by simply using an option in your LaTeX IDE menu TeXcount runs and the output is opened in your browser. I normally set this to a keyboard shortcut, so I don’t even need to use a menu.

Simple!

Effectively using MATLAB with Dropbox

I love dropbox. I love being able to have all of my files on every computer I use, both personal and work machines. And overtime I have found some useful fixes and hacks to allow an even more seamless transition from one computer to another. This normally involves linking configuration files, however for some programs a bit more is needed. One example is MATLAB.

I find MATLAB a really useful environment for data analysis, calculations, and performing a number of repetitive tasks. As part of this I have written a number of my own functions which I use regularly. Secondly, I often have programs which load in data from other files stored in my dropbox, so the full path is different on different machines. Using relative paths is only a half fix as I don’t always run programs in their folders. Also my home computers are now on linux (Ubuntu and Lubuntu) while my work machine is on Windows. So here is how I have configured my MATLAB.

Setting the same starting directory.

I have a folder called MATLAB_home in my dropbox. So if I am ever lazy and write a quick program and save it in the local directory it will always be in dropbox and in the local directory when I open MATLAB elsewhere.

To set the starting directory in Windows you simply right click on the MATLAB icon, select properties, and then enter the full path to your MATLAB_home folder in dropbox. Then click OK.

In Ubuntu/Lubuntu I use MainMenu (a useful program for controlling menu items) to change or (or create if it was not created during install) the MATLAB launcher. In the section labelled command I have changed the command to (%s are part of the path, don’t add %s):
sh -c "cd %PathToMatlabHomeFolder% &&%PathToMatlabExecutable% -c %PathToMatlabLicenceFile% -desktop"

This needs to be done on each machine.

Set up a functions directory

Now, over time you have probably written, been sent, and downloaded a number of MATLAB functions which you use regularly. I know many people who simply copy these to any new working directory leaving them with multiple copies of the same files in many locations. This is not only time consuming, it is also a silly idea, if you ever update anything in these functions then everything becomes out of sync.

As many of you know MATLAB has a path, similar to the path in unix or windows. For those who don’t know what this means. Essentially any time you call a function MATLAB checks through every folder in the path to see if that folder has the function you called.

So now make another directory in dropbox (or you can just use your MATLAB_home directory if you prefer). I called mine MATLAB_ROOT. Now we need to run MATLAB as an administrator/superuser if we are to permanently change the path (In windows XP or before you don’t need to run as an administrator).

Once MATLAB opens. Select “Set Path…” from the File menu. In select “Add Folder” and select your new functions folder. Click “OK”. Then in the Set Path window click “Save”, then “Close”.

Again you need to set the path on every computer you use. Now you can put your own functions in this folder and use them from any folder on any of your computers.

Dropbox friendly file paths in MATLAB

So, now imagine you are writing a program which opens files from a number of different folders in your dropbox. I do this by making a new function which returns my dropbox path for the computer I am on. I identify my computers using their names, MATLAB can get the name of a computer by using the following command
[blank, SYSname] = system('hostname');
which will save the computer’s name to “SYSname” and a 0 to “blank”, (in newer versions of MATLAB blank can be replaced by a ~ character as it is not needed). The SYSname variable returned always has an extra character on the end which displays as a space but screws up string matches, so we ignore it.

Now make a new file an paste in the following code:

function path = DropboxPath

[blank, SYSname] = system('hostname');

if strcmp(SYSname(1:end-1),'Oliphant')
    path = '/home/gobaskof/Dropbox';
elseif strcmp(SYSname(1:end-1),'Prometheus')
    path = 'G:\Documents\Dropbox';
else
    error('Computer not recognised')
end

This is an example with two of my computers (Oliphant is a very large laptop, Prometheus is my powerful work PC, but the university IT people regularly metaphorically pick out its liver.). Replace the name with the names of your computers, and the paths with the paths to your dropbox folder on that computer. Repeat the elseif block for any extra computers.

Now save this file as DropboxPath.m in your newly created functions folder.

So now if you want to open the file ‘file.dat’ which is in the folder ‘Data’ of your dropbox you can simply run:
FID = fopen([DropboxPath,'\Data\file.dat'],'r');
This will new work on any of your (Windows) computers. Replace / with \ and it will work on any of your UNIX computers. (Below I will discuss how to make thing run if you use both Windows and UNIX).

At this point it is worth mentioning, if you ever want to output things in a your computer’s temporary folder you come to a similar problem. MATLAB has a function for this called ‘tempdir’ which returns the current computer’s temporary folder.

Paths which work in UNIX and Windows

Paths in unix use / to separate folders, Windows uses \, so this can be a problem. It is possible to get around this by using the Matlab command ‘fullfile’, which accepts string inputs for every folder and the file and returns a string with them all linked together with the correct file separator. i.e.:
fullfile('Data','file.dat')
returns
Data/file.dat
For long file paths this is a ball-ache. Also the MATLAB function ‘filesep’ comes in handy, it simply returns / in UNIX and \ in Windows, but again always using this in every file path is a annoying. So instead I have made another simple function (again saved in the new function directory in dropbox), this one corrects all file paths to the current operating system:

function path = CorrectFilePath(path)

if isunix
    pos = strfind(path,'\');
    path(pos) = '/';
else
    pos = strfind(path,'/');
    path(pos) = '\';
end

Once this is saved in your functions folder you can use the following syntax to open a file and it will work on both UNIX and Windows computers:
FID = fopen([DropboxPath,CorrectFilePath('\Data\file.dat')]),'r');
which is much more convenient for files buried many folders deep in your dropbox as you can just copy the path on whichever computer you are using at the time.

That’s all folks

And that’s it. Using the same MATLAB code on multiple computers should now be pain-free.

SPIW toolbox released.

After years of using free software and singing the praises of the open source community, I have finally contributed by open sourcing a MATLAB toolbox which I co-wrote with other members of the Nottingham nanoscience group.

The toolbox is called SPIW, standing for: scanning probe image wizard (See also my local page on SPIW). It can be used to automate many image analysis tasks which scanning probe microscope (SPM) users regularly do by hand. The idea is that by opening up the code to the community it will help many SPM users, and that others will contribute to make SPIW better.

One could argue that writing an open source toolbox for a proprietary computing environment does not fully agree with the goals of the open source community. However, MATLAB is used by most universities across the world, and makes it very simple to modify and improve code compared to compiled programming languages. Perhaps the future will bring a C++ SPIW dll, or maybe an Octave port of SPIW, but currently I am proud to announce SPIW for MATLAB.

Ubuntu/Lubuntu on a Samsung NC10

My old trusty NC10, seemed to be on its last legs. Slowly loading web pages on windows xp, not quite being able to handle multiple programs any more, especially not anything graphical such as GIMP. I thought I would give it a go on Ubuntu 12.10, which runs so well on my main laptop. It did run faster with Ubuntu, but still things were still slow and frustrating. Looking around on the web many people commented on how well their NC10 runs Ubuntu, but most of these posts were about Ubuntu 9 or 10. Looking at the task manager the limiting seems to be the little Atom processor, not the 1GB of RAM.

I have since installed Lubuntu 13.04, which is a lightweight Ubuntu which uses a LXDE interface. My initial impressions of Lubuntu are mixed, it looks pretty rather ugly compared to Ubuntu (which I run with gnome desktop as I find unity awful), I am also less impressed with the task manager, file manager, and keyboard shortcut management; but it does run so quickly and smoothly. It is so quick and smooth that I don’t think I will be returning to Ubuntu on the NC10. I think I will just accept that a four an a half year old netbook needs a lightweight operating system, even if it is less polished.

My advice for any fellow NC10 users getting frustrated with how slow it has got is: try Lubuntu.

Tetr on 64bit ubuntu

I recently started using tetr, a user friendly program for setting up DFT calculations among other things. Tetr, has to be built on the users (unix) machine. A makefile is available, however, the objects used for the build are 32-bit fortran objects. This lead me into some problems with the build. I thought I would provide my solution here just in case anyone else is having the same issue.

My solutions was to just compile using the gfortran compiler (which can be installed from the Ubuntu software centre) using a 32-bit flag to force a 32 bit compile. However there are some issues with libraries which need to be solved. First you need to install some new 32 bit libraries with:

sudo apt-get install ia32-libs
sudo apt-get install lib32gcc1 libc6-i386 lib32z1 lib32stdc++6
sudo apt-get install lib32asound2 lib32ncurses5 lib32gomp1 lib32z1-dev lib32bz2-dev
sudo apt-get install g++-multilib
sudo apt-get install libgfortran3:i386

Then set up a coulple of symbolic links:
sudo  ln -s /usr/lib32/libstdc++.so.6 /usr/lib32/libstdc++.so
sudo ln -s /usr/lib/i386-linux-gnu/libgfortran.so.3 /usr/lib/i386-linux-gnu/libgfortran.so

Then update LD_LIBRARY_PATH in /etc/profile with:
sudo gedit /etc/profile &
Add these 2 lines to the end of the file:
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib32:/usr/lib32:/use/lib/i386-linux-gnu
export LD_LIBRARY_PATH

Save and exit gedit.

Finally replace the make file with THIS make file, and make like usual.

Happy simulating!