Adobe has some documentation concerning start parameters for PDFs here and here.

Open via chapter name

Append the complete chapter name to your link URL.

Example: http://blog.dispatched.ch/sps/thesis.pdf#3.4%20Programmiersprache%20und%20Paradigmen

Note: Chapter names are to be encoded as is common practice in URLs. In this example blanks are encoded to %20.

Open a specific page

Append the attribute ‘page’ followed by the page number to your link URL.

Example: http://blog.dispatched.ch/sps/thesis.pdf#page=38

Since the project is in stealth mode right now, I can’t say which pages I am referring to, but let me give you these facts:

• ~170kb HTML code
• W3C validation shows about 1300 errors and 2600 warnings per page

Considering this many errors and warnings, I previously thought the job had to be done using BeautifulSoup, because it is known to have a very error resistant parser. In fact, BeautifulSoup doesn’t parse the HTML directly, but splits the tags in tag-soup by applying regular expressions around them. Opposing popular stories this seems to make BeautifulSoup very resilient towards bad code.

However, BeautifulSoup doesn’t perform well on the described files. The task: I need to parse 20 links of a particular class off the page. I put the relevant code in a seperate method and profiled it using cProfile:

cProfile.runctx("self.parse_with_beautifulsoup(html_data)", globals(), locals())

def parse_with_beautifulsoup(html_data):
  soup = BeautifulSoup.BeautifulSoup(html_data)
  links_res = soup.findAll("a", attrs={"class":"detailsViewLink"})
  links = [car_link["href"] for car_link in car_links_res]

Parsing 20 pages, this takes 167s on my small Debian VPS. Thats 8s+ per page. Incredibly long. Thinking of how BeautifulSoup parses, it’s understandable however. The overhead of creating tag-soup and parsing via RegExp leads to a whopping 302’000 method calls for just these four lines of code. I repeat: 302’000 method calls for four lines of code.

Hence, I tried lxml. The corresponding code is:

root = lxml.html.fromstring(html_data)
links_lxml_res = root.cssselect("a.detailsViewLink")
links_lxml = [link.get("href") for link in links_lxml_res]
links_lxml = list(set(links_lxml))

On the 20 pages, this takes only 2.4s. That’s only 0.12s per page. lxml needed only 180 method calls for the job. It runs 70x faster than BeautifulSoup and creates 1600x fewer calls.

When you do a graph of these numbers, the performance difference looks ridiculous. Well, let’s have some fun(;

lxml vs BeautifulSoup performance

Considering lxml supports xpath as well, I’m permanently switching my default HTML parsing library.

Note: Ian Bicking wrote a wonderful summary in 2008 on the performance of several Python HTML parsers which led me to lxml and to this article.

Update (08/17/2010): I planned on implementing my results on Google AppEngine. “Unfortunately” lxml relies heavily on C-code (that’s where the speed comes from^^). AppEngine is a pure Python environment. It will never run modules written in C.

There’s much joy to extract from this graph from WolframAlpha (note: WA uses data from 06/30/2010. The breakthrough was imminent back then):

Now, please don’t come bothering me, because Microsoft still has Windows and Office to receive the big routine paycheck while Apple has to constantly innovate to keep its pace. Depending on those two has thought Microsoft nothing but stagnation. In the meantime Apple, Google, Facebook and Twitter arose. If Microsoft doesn’t change pace, they will suffer Yahoos’ fate.

If you ask me, it would be for the best of it. In the field of technology not the biggest player will win in the long run, but the most innovative. Do you remember those Kodak cameras that didn’t go digital (1990:7B, 2010:0$, source)? Geocities was hip once and worth 3.6B in 1999, now dead (source).

The list continues. Even when there is market domination, there still is room to manoeuvre. Microsoft didn’t equip the first personal computers – and they certainly won’t be the last.

I won’t elaborate on how to get an account, download the SDK and get started, because Google hosts great tutorials for these itself. If you are already familiar with Python web development this will get you started in a matter of minutes.

I personally chose not to use the Google webapp framework, because I’m quite familiar with CherryPy. I fell in love with it, because it feels very sleek – very Zen-like. This comes to no surprise, because it was a deliberate design decision as can be read in The Zen of CherryPy.

Getting started with CherryPy on GAE is no trouble, either. GAE supports any Python framework that is WSGI-compliant. Those include Django, CherryPy, web.py and Pylons. Google doesn’t host these frameworks themselves, so all you have to do is copy the whole framework into your GAE project to get the import to work. That’s it. Same counts for any 3rd party module. Need BeautifulSoup? Just copy the py-file to your project. Easy as cake.

Now, if you want to serve images dynamically, you don’t have to store them on harddisk to link to them. Just save them in the Google Datastore and serve whenever needed.

Using the following snippet you will be able to dynamically serve images with URLs like this:

http://application/handler_name/index/[0-9]*

import cherrypy
from cherrypy import expose
import wsgiref.handlers
import DynamicImage

class Root:
  @expose
  def index(self):
    return ""

class GetImage():
  """ GetImage provides a handler for dynamic images """

  def __init__(self):
    """
      Mockup for getting some images. Datastore or live
      scraping could be done here
    """
    # Note: DynamicImage is just a mockup.
    # There is no such module.
    dynamic_image = DynamicImage.DynamicImage()
    self.pictures = dynamic_image.getImages()

  @expose
  def index(self, num=None):
    """
      Provides the handler for urls:
        application/handler_name/index/[0-9]*
    """
    return self._convert_to_image(self.pictures[0][int(num)])

  def _convert_to_image(self, picture):
    cherrypy.response.headers['Content-Type'] = "image/jpg"
    return picture

# Root() doesn't do anything here. It normally serves your index page.
root = Root()

# Generate route http://app/img/
root.img = GetImage()

# Start CherryPy app in wsgi mode
app = cherrypy.tree.mount(root, "/")
wsgiref.handlers.CGIHandler().run(app)

One last note: Processes running longer than 15-30s will be cut off from GAE with the DeadlineExceededError exception. You can catch this exception and try to divide your workload into smaller pieces.

Anyway, I normally don’t do any administration using web interfaces (such as Confixx, Plesk and the like), but I have come across Webmin, which is “a web-based interface for system administration for Unix. Using any modern web browser, you can setup user accounts, Apache, DNS, file sharing and much more.”

Webmin supports user administration. Any account can implement the role of “Change Passwords”. Unfortunately, this will enable this particular user to change any other users password. Lucky for me, Webmin is OSS and written in Perl. So a quick four line code change does exactly what I need.

This is how the Munin front-end looks after the patch. If you’re not root, you can only change your own password. End of line.

If you want to recreate this setup on your machine, you can download the patch here and apply it with

$ patch /usr/share/webmin/passwd/index.cgi > index.patch

Have fun.

Thunderbird is a good basis and does the job well. It’s sleek and has good IMAP support. Combine that with plugins for VCS support, Lightning for an integrated calendar and the provider for Google Calendar, you find yourself with a decent toolset. What kept bugging me is editing the mails.  Coming from mutt/VIM, I might be biased on that one. Heck, I’m even using the Vimperator plugin in Firefox and find that it brightens each and every day.

Anyway, there is release to that pain! There’s a plugin called “External Editor” – it works in Windows as it does in real OSs and it’s actually quite a charm as you can see. You can find all you need on globs.org. Just follow the instructions and you’ll be happy.

Tips: Customize your mail view to show the feature “External Editor” or use the pre-defined shortcut CTRL+e to open your custom editor (that is VIM for me^^).

Have fun and enjoy the sweet life(;   ]]> http://blog.dispatched.ch/2010/06/30/vim-to-write-mails-in-thunderbird/feed/ 1 http://blog.dispatched.ch/2010/05/10/c-40s-dynamicity/ http://blog.dispatched.ch/2010/05/10/c-40s-dynamicity/#comments Mon, 10 May 2010 21:23:21 +0000 Alain M. Lafon http://blog.dispatched.ch/?p=985 I just found an article ranking highly on Hacker News (my favourite read) concerning the release of C# 4.0 – you can find it on blogs.msdn.com. On this blog Microsoft claims a couple of highly sophistacated new features. Being the spoiled guy I am, they just seem natural to me. Since they started their article with the words “The dynamic keyword is a key feature of this release.”, let me demonstrate the new features in a really dynamically typed language where they have been around for quite some time: Python

Dynamic

                        dynamic contact = new ExpandoObject();
                        contact.Name = "Patrick Hines";
                        contact.Phone = "206-555-0144";
                        

                        class contact: None
                        contact.Name = "Patrick Hines"
                        contact.Phone = "206-555-0144"
                        

Optional (or Default) Parameters

                        public static void SomeMethod(int optional = 0) { }
                        SomeMethod(); // 0 is used in the method.
                        SomeMethod(10);
                        

                        def some_method(optional = 0): pass

                        some_method()
                        some_method(10)
                        

Named Arguments

                        var sample = new List<String>();
                        sample.InsertRange(collection: new List<String>(), index: 0);
                        sample.InsertRange(index: 0, collection: new List<String>());
                        

                        def foo(bar, foobar): None

                        foo(bar='asdf', foobar=12)
                        foo(foobar=12, bar='asdf')
                        

I honestly know that a comparison of a statically typed language on the CLR and an interpreted dynamic language doesn’t account for too much. But since Microsoft is making a fuzz about dynamic being the keyword of the new release, I felt the urge to drop this note.

The Python code was tested with v2.5 – that’s the oldest installation I’ve got. However, it’s old enough, because .NET didn’t even have decent IPC back then (i.e. Named pipes were added in .NET 3.5).

Well, that’s my rant for the night – I’m going back to teaching myself a real programmers language(Clojure) – as you should, too(;

If you liked the article, follow me on twitter here

First of all, I’m defining the used nomenclature:

• ASCII characters are being written in single quotes
• decimal numbers are of the type Long with a L suffix
• hex values have a x prefix

First, let me quote the documentation:

I’ll begin with hexlify(). As the documentation states, this method splits a string which consists of hex-tuples into distinct bytes.

The ASCII character ‘A’ has 65L as numerical representation. To verify this in Python:

>>> long(ord('A'))
65L

You might ask “Why is this even relevant to understand binascii?” Well, we don’t know anything about how ord() does its job. But with binascii we can re-calculate manually and verify.

>>> binascii.hexlify('A')
'41'

Now we know that an ‘A’ – interpreted as binary data and shown in hex – resembles ’41′. But wait, ’41′ is a string and no hex value! That’s no biggy, hexlify() represents its result as string.

To stay with the example, let’s convert 41 into a decimal number and check if it equals 65L.

>>> long('41', 16)
65L

Tada! It seems that ‘A’ = 41 = 65L.
You might have known that already, but please, stay with me a minute longer.

To make it look a little more complex:

>>> binascii.hexlify('A') == "%X" % long('41', 16)
True

Be aware that

>>> "%X" %n

converts a decimal number into its hex representation.

——

binascii.unhexlify() naturally does the same thing as hexlify(), but in reverse. It takes binary data and displays it in tuples of hex-values.

I’ll start off with an example:

	>>> binascii.unhexlify('41')
	'A'

	>>> binascii.unhexlify("%X" % ord('A'))
	'A'

Here, unhexlify() takes the numerical representation 65L from the ASCII character ‘A’

	>>> ord('A')
	65

converts it into hex 41

	>>> "%X" % ord('A')
	'41'

and represents it as a 1-tuple (meaning dimension of one) of hex values.

And now the conclusio – why might all of this be useful?
Right now, I can think of at least four use cases:

• cryptography
• data-transformation (i.e. Base64 for MIME/E-Mail attachements)
• security (deciphering binary readings off a network, pattern matching, …)
• textual representation of escape sequences

Taking up the last example, I’ll show you how to visualize the Bell esape sequence (you know, that thing that keeps beeping in your terminal).
Taken from the ASCII table, the numerical representation of the Bell is 7. Programmers might know it better as a.

	>>> '7' == 'a'
	True

Presuming you read such a character in some kind of binary data – for example from a socket

	>>> foo = '7'

and you want to visualize this data

	>>> print foo

you will not get any results – at least none visible. You might hear the Bell sound if you’re not on a silent terminal.

Now, finally – binascii to the rescue:

	>>> binascii.hexlify('7')
	'07'

Voilà, the dubious string is decrypted.

Now, if your victim wants to forward your mail, Lotus Notes would respond with a little pop-up:

This certainly looks like a magical and proprietary feature, doesn’t it?  Let’s look at the source of such a “mail”(aka memo in Notus’ language) – you will have to forward it to another mail-client though, because memos can’t be displayed in source:

...
Subject: Testnachricht
MIME-Version: 1.0
Sensitivity: Private
X-Mailer: Lotus Notes Release 6.5.5  CCH1 March 07, 2006
...

As you can see, there is a proprietary meta-flag Sensitivity: Private. It can be reproduced with any decent mail user agent or programmatically. What follows is a little Python code snippet that just does the trick:

import smtplib
from email.message import Message
msg = Message()
msg.set_payload("Testmessage Body")
msg["Subject"] = "Testmessage from Python"
msg["From"] = "preek@dispatched.ch"
msg["To"] = "somebody@somewhere.com"
msg["Sensitivity"] = "Private"
smtp = smtplib.SMTP("localhost")
smtp.sendmail("preek@dispatched.ch", "somebody@somewhere.com", msg.as_string())

But please, don’t use this information unless you absolutely have to. Lotus Notes.. *brr*.

Enjoy(;

If you liked this article, please feel free to re-tweet it and let others know.

With pTwittery.com you may export all your tweets (aka timeline) into a diary. Your personal diary will then be sent to you via e-mail.

Enjoy, everyone(;

If you liked this article, please feel free to re-tweet it and let others know.

This is where we are going:

So, these are my thoughts on a VIM setup for coding (Python).

Modern GUI VIM implementations like GVIM or MacVIM give the user the opportunity to organize their open files in tabs. This might look convenient, but to me it is rather bad practice, because a second tab will not be in the in the same buffer scope as the first one which takes away from future interaction options between the two. Using MiniBufExplorer, however, gives the user tabs(not only in the GUI, but also in command line) and leaves the classic buffer interaction intact.

Being able to neatly work on multiple files, the user still misses the potential his favourite IDE gives him in visualizing classes, functions and variables. Luckily there are quite a few plugins around to accomplish this task just as well. My favourite one would be TagList. TagList uses Exuberant Ctags for actually generating the tags(note: it really relies on this specific version of ctags – preinstalled implementations on UNIX systems won’t work).

A lot of coders have the habit of using TODO or FIXME statements in their code. Other IDEs often rely on having good third party project management software, but not VIM. There are great plugins like Tasklist reminding the programmer of those lines of code. Tasklist even implements custom lists – to me that’s an incredible productivity gain.

In these times, the programmer knows his or her programming language more or less by interactively finding out what it can do. Therefore code completion(sometimes also called IntelliSense*ugh*) is a major feature. I have heard  many people saying that this is where VIM fails – but luckily they are plain wrong(; In V7, VIM introduced omni completion – given it is configured to recognize Python (if not, this feature is only a plugin away) Ctrl+x Ctrl+o opens a drop down dialog like any other IDE – even the whole Pydoc gets to be displayed in a split window.

Probably the most wanted feature(besides code completion) is debugging graphically. VimPDB is a plugin that lets you do just that(. I acknowledge it is no complete substitution for a full fledged graphical debugger, but I honour the thought that having to rely on a debugger (often), is a hint of bad design.

–

From the eye-candy to the implementation. Don’t worry, it’s no sorcery.

First of all, make sure you have VIM version 7.x installed, compiled with Python support. To check for the second, enter :python print “hello, world” into VIM. If you see an error message like “E319: Sorry, the command is not available in this version”, then it’s time to get a new one. If you’re on a Mac, just install MacVIM(there’s also a binary for the console in /Applications/MacVim.app/Contents/MacOS/). If you’re on Windows, GVIM will suffice(for versions != 2.4 search for the right plugin). If you’re on any other machine, you will probably know how to compile your very own VIM with Python support.

Second, check if you have a plugin directory. In Unix it would typically be located in $HOME/.vim/plugin, in Windows in the Program Files directory. If it doesn’t exist, create it.

Now, let’s start with the MiniBufExplorer. Get it and copy it into your plugin directory. To start it automatically when needed and be able to use it with keyboard and mouse commands, append these lines in your vimrc configuration:

let g:miniBufExplMapWindowNavVim = 1
let g:miniBufExplMapWindowNavArrows = 1
let g:miniBufExplMapCTabSwitchBufs = 1
let g:miniBufExplModSelTarget = 1

For a project view, get TagList and Exuberant Ctags. To install Ctags, unpack it, go into the directory and do a compile/install via:

./configure && sudo make install

Ctags will then be installed in /usr/local/bin. When using a Windows machine, I recommend Cygwin with GCC and Make; it’ll work just fine. If you don’t want to tamper with your original ctags installation, you can propagate the location to VIM by appending the following line to vimrc:

let Tlist_Ctags_Cmd='/usr/local/bin/ctags'

To install TagList, just drop it into VIMs plugin directory. You will now be able to use the project view by typing the command :TlistToggle.

Tasklist is a simple plugin, too. Copying it into the plugin directory will suffice. I like to have shortcuts and have added

map T :TaskList<CR>
map P :TlistToggle<CR>


to vimrc. Pressing T will then open the TaskList if there are any tasks to process. q quits the TaskList again.

VimPDB is a plugin, as well. Install as before and see the readme for documentation. If it doesn’t work out of the box, watch for the known issues.

To enable code(omni) completion, add this line to your vimrc:

autocmd FileType python set omnifunc=pythoncomplete#Complete

If it doesn’t work then, you’ll need this plugin.

My last two recommondations are setting these lines to comply to PEP 8(Pythons’ style guide) and to have decent eye candy:

set expandtab
set textwidth=79
set tabstop=8
set softtabstop=4
set shiftwidth=4
set autoindent
:syntax on

There are certainly a lot more flags to help productivity, but those will probably be more user specific.

Have fun coding Python while not being bound to a specific IDE, but having all the benefits of VIM bundled with a few helping hands. Enjoy, everyone.

If you liked this article, please feel free to re-tweet it and let others know.

You should follow me on twitter here

1. Get, compile and install Python (I have used version 2.5.4)
   • http://www.python.org/download/releases/
   • unpack
   • make sure you have a recent version of GCC installed
   • ./configure && make && make install
   • as a result Python will be installed in /usr/local
2. Get, compile and install Setuptools
   • http://pypi.python.org/pypi/setuptools
   • unpack
   • python setup.py install

3. Get, compile and install pysqlite
   • http://oss.itsystementwicklung.de/trac/pysqlite/wiki/WikiStart#Downloads
   • unpack
   • add line “library_dirs=/usr/local/lib” to pysqlite-x.y.z/setup.cfg
   • globally export your library paths:
   • LD_LIBRARY_PATH=/opt/csw/lib/:/usr/lib/:/lib/:/usr/local/lib:$LD_LIBRARY_PATH ; export LD_LIBRARY_PATH
   • python setup.py install
4. easy_install install sqlalchemy
5. easy_install jinja2
6. Get, compile and install Juno
   • http://brianreily.com/project/juno
   • python setup.py install

Enjoy.

In this short tutorial, I’m going to show you how to scrape a website with the 3rd party html-parsing module BeautifulSoup in a practical example. We will search the wonderful translation engine dict.cc, which holds the key to over 700k translations from English to German and vice versa. Note that BeautifulSoup is liscensed just like Python while dict.cc allows for external searching.

First of, place BeautifulSoup.py in your modules directory. Alternatively, if you just want to do a quick test, put in the same directory where you will be writing your program. Then start your favourite text editor/Python IDE(for quick prototyping like we are about to do, I highly recommend a combination of IDLE and VIM) and begin coding. In this tutorial we won’t be doing any design; we won’t even encapsulate in a class. How to do that, later on, is up to your needs.

What we will do:

1. go to dict.cc
2. enter a search word into the webform
3. submit the form
4. read the result
5. parse the html code
6. save all translations
7. print them

You can either read the needed coded on the fly or download it.
Now let’s begin the magic. Those are our needed imports.

import urllib
import urllib2
import string
import sys
from BeautifulSoup import BeautifulSoup

urllib and urllib2 are both modules offering the possibility to read data from various URLs; they will be needed to open the connection and retrieve the website.  BeautifulSoup is, as mentioned, a html parser.

Since we are going to fetch our data from a website, we have to behave like a browser. That’s why will be needing to fake a user agent. For our program, I chose to push the webstatistics a little in favour of Firefox and Solaris.

user_agent = 'Mozilla/5 (Solaris 10) Gecko'
headers = { 'User-Agent' : user_agent }

Now let’s take a look at the code of dict.cc. We need to know how the webform is constructed if we want to query it.

...
<form style="margin:0px" action="http://www.dict.cc/" method="get">
  <table>
    <tr>
      <td>
        <input id="sinp" maxlength="100" name="s" size="25" type="text" />
        style="padding:2px;width:340px" value="">
      ...</td>
    </tr>
  </table>
</form>
...

The relevant parts are action, method and the name inside the input tag. The action is the webapplication that will get called when the form is submitted. The method shows us how we need to encode the data for the form while the name is our query variable.

values = {'s' : sys.argv[1] }
data = urllib.urlencode(values)
request = urllib2.Request("http://www.dict.cc/", data, headers)
response = urllib2.urlopen(request)

Here the data get’s encapsulated in a GET request and packed into the webform. Notice that values is a dictionary which makes handling more complex forms a charm. The the form gets submitted by urlopen() – i.e. we virtually pressed the “Search”-button.
See how easy it is? These are only a couple lines of code, but we already have searched on dict.cc for a completely arbitrary word from the commandline. The response has also been retrieved. All that is left, is to extract the relevant information.

the_page = response.read()
pool = BeautifulSoup(the_page)

The response is read and saved into regular html code. This code could now be analyzed via regular string.find() or re.findall() methods, but this implies hard-coding in reference to a lot of the underlying logic of the page. Besides, it would require a lot reverse engineering of the positional parameters, setting up several potentially recursive methods. This would ultimately produce ugly(i.e. not very pythonic) code. Lucky for us, there already is a full fledged html parser which allows us to ask just about any generic question. Let’s take a look at the resulting html code, first. If you are not yet familar with the tool that can be seen in the screenshot; I’m using Firefox with the Firebug addon. This one is very helpful if you ever need to debug a website.

Let me show an excerpt of the code.

<table>..
  <td class="td7nl" style="background-color: rgb(233, 233, 233);">
    <a href="/englisch-deutsch/web.html">
      <b>web</b>
    </a>
  </td>
<td class="td7nl" ... /td>
</table>..

The results are displayed in a table. The two interesting columns share the class td7nl. The most efficient way would seem to just sweep all the data from inside the cells of these two columns. Fortunately for us, BeautifulSoup implemented just that feature.

results = pool.findAll('td', attrs={'class' : 'td7nl'})
source = ''
translations = []

for result in results:
    word = ''
    for tmp in result.findAll(text=True):
        word = word + " " + unicode(tmp).encode("utf-8")
    if source == '':
        source = word
    else:
        translations.append((source, word))

for translation in translations:
    print "%s => %s" % (translation[0], translation[1])

results will be a BeautifulSoup.ResultSet. Each member of the tuple is the html code of one column of the class td7nl. Notice that you can access each element like you would expect in a tuple. result.findAll(text=True) will return each embedded textual element of the table. All we have to do is merge the different tags together.
source and word are temporary variables that will hold one translation in each iteration. Each translation will be saved as a pair(list) inside the translations tuple.
Finally we iterate over the found translations and write them to the screen.

$ python webscraping_demo.py
 kinky   {adj} =>  9 kraus   [Haar]
 kinky   {adj} =>  nappy   {adj}   [Am.]
 kinky   {adj} =>  6 kraus   [Haar]
 kinky   {adj} =>  crinkly   {adj}
 kinky   {adj} =>  kraus
 kinky   {adj} =>  curly   {adj}
 kinky   {adj} =>  kraus
 kinky   {adj} =>  frizzily   {adv}

In a regular application those results would need a little lexing, of course. The most important thing, however, is that we just wrote a translation wrapper onto a webapplication – in only 28 lines of code. Did I mention that I’m in love with Python?

All that is left is for me to recommend the BeautifulSoup documentation. What we did here really didn’t cover what this module is capable of.

I wish you all the best.

So I started working on my very own plugin that would help reduce writing redundant code over and over again. I have hosted a demo version right here which should look like this:

As you can see the interface really is straight forward; click the title to sort the column – and click a number or an arrow to move from page to page.

The nice thing is that this implementation should work with about any database format; and any SQL or ActiveRecord statement. Let me show you how much code you’d have to write to implement the table from above:

In the controller you’d fetch the data with something like:

session[:data] = StUser.find(:all, :limit => 600)

or even something as ugly as

session[:data] = StUser.find_by_sql("select top 600 * from st_users")

Now to the interesting part – in the view you will only have to write one single line; p_pagination is generic – meaning it automatically adjusts to the number of columns and the input data types.

<%= p_paginate(session[:data], {:max => 10, :page => session[:page].to_i},
"login", "MAIL", "FIRSTNAME", "LASTNAME", "id") %>

Just give it the database resultset(session[:data]), the number of rows per page and the attributes that are to be read from the resultset.

Now you have seen the might and the first drawback of p_pagination. All data is currently saved in the session, which means that the whole resultset is held available – many of you will now think this is very inefficient and draws performance and memory and I would be inclined to support your thought; only if databases like MSSQL supported the allmighty LIMIT and OFFSET commands like MySQL does. Since it doesn’t, I have deliberately chosen this path, because the needed workarounds in MSSQL like select top 10 * from users where login not in (select top 5 login from users) will result in a massive table search – and if your legacy database works on four text columns that alltogether emulate a primary key in userspace.. then you will really blast your performance away.
Anyway – storing the resultset in the session is not that bad, either, because RoR supports storing session information back in the database as explained in http://wiki.rubyonrails.org/rails/pages/HowtoChangeSessionStore. However even in the database the space for each session is limited, so this current implementation won’t perform on massive data with thousands of rows – but on smaller data collections it does the job pretty well as you can see.

There’s one more drawback; my code doesn’t seem to work on all browsers properly – I designed it using Firefox3 and IE6 where it just works fine. But I have seen that Safari displays the spinner to the right of the pagination bar while Firefox2 will scramble the bar and render it useless; Opera I haven’t tried yet.

So this is it – I’d like to call this a preview alpha release even though it already went productive on some small systems where it does its job as expected.

In a following post I will explain how to install p_pagination as a plugin.

! LaTeX Error: Command \Telefon already defined.Or name \end... illegal, see p.192 of the manual.

See the LaTeX manual or LaTeX Companion for explanation.Type  H   for immediate help. ...

l.48 \newcommand\Telefon{\mvchr{84}}

This bug is in g-brief with newer versions of marvosym.sty. So all you have to do is edit line 48 of /texlive/texmf-dist/tex/marvosym.sty and uncomment it.

This bug might affect all unices, not only Solaris.

6      0       US6.kt

and rewrite it to your favourite layout(British English in this case)

6       0      UK6.kt

There is one drawback however; from this point on this is going to be the single standard fallback layout for all keyboards hooked via SUN Ray clients to your server which won’t report a valid layout themselves. So if you want to use different layouts on different clients you might want to check the xkb option in “/opt/SUNWut/bin/utxconfig”, which I haven’t done until now.

Why would somebody want to let postfix connect to a SQL-database?

• There’s no need to create a real local user for each e-mail account
• SQL-databases can be kept in RAM, so if you have excessive mailing
  on your server, there will be reduced harddisk access
• Management of mailinglists becomes real easy
• /etc/aliases is kept small and simple

Step 1
Install the package “mysql-server” and “mysql-client” if not yet installed.
Log on to your sql-server using the root account:

mysql --user root
mysql> create database postfix_database;
mysql> GRANT ALL PRIVILEGES ON postfix_database \
TO 'postfix'-AT-'localhost' IDENTIFIED BY 'postfix_password' \
WITH GRANT OPTION;
mysql> flush privileges;
mysql> create table postfix.postfix_alias (destination VARCHAR(50), \
alias VARCHAR(50));
mysql> exit;

Now we have created a database called “postfix_database” and a user called
“postfix” who has access to it using his unique password “postfix_password”.
With “flush privileges” we bring the sql-server up to date concerning user rights.
Then we create a table called “postfix_alias” in the database “postfix” with two rows:
“destination” is a text variable where the mail will be relayed to and “alias” is the name
of the mailinglist in my example.

Step 2
Install the package “postfix-mysql”. Besides the needed
library this will bring you the config file “/etc/postfix/mysql-aliases.cf” which we
will modify like this

user = postfix
password = postfix_password
table = postfix_alias
query =  SELECT destination FROM postfix_alias WHERE alias = '%s'
hosts = unix:/var/run/mysqld/mysqld.sock
select_field = destination
where_field = alias

Since postfix runs in a chroot it lacks several information it needs to have;
for example the socket to the mysql daemon. That’s why we provide it
with some bind mounts, which can be done by inserting these lines into
“/etc/fstab”.

/etc/passwd     /var/spool/postfix/etc/passwd           none bind 0 0
/etc/shadow     /var/spool/postfix/etc/shadow           none bind 0 0
/etc/group      /var/spool/postfix/etc/group            none bind 0 0
/var/run/mysqld /var/spool/postfix/var/run/mysqld       none bind 0 0

To update this information the root user has to remount all filesystems
using “mount -a”.

Step 3
We’re done already(almost). All that is still needed is some information in the database.
Single entries can be made with the mysql client like this:

mysql> insert into postfix_alias values \
('someone-AT-somewhere-DOT-de', 'mailinglistname');

Now if you send a mail to “mailinglistname-AT-yourhost-DOT-com” the mail will be relayed to
“someone@somewhere-DOT-de”. That’s it.
I wrote a JSP/Servlet combination in JavaEE to create a webpage where users can
put themselves on or off a mailinglist; you can find it
here or in the
projects folder if you’re interested.

Step 4

Note that installing the package postfix-mysql updated a line in your “/etc/postfix/main.cf”:

alias_maps = hash:/etc/aliases
...
alias_maps = mysql:/etc/postfix/mysql-aliases.cf

There are most likely many more lines in this file, but the important factor is
that the first line mapping to “/etc/aliases” is made obsolete by the second entry.
So if you were using some important relaying in this file you should migrate it.
For this reason I wrote a small
shellscript that was capable to do the job for my setup.

• Change the “Device” section that it looks like this:

  Section "Device"
      Identifier              "Videocard0"
      Driver                  "nvidia"
      # optional (find out with "$ Xorg -scanpci")
      BusID                  "[Your BusID, e.g.: PCI:2:0:0]"
      Screen                0
  EndSection
  

• Add a new “Device” section for the TV, just like the one before, but change “Screen 0″ to “Screen 1″ and “Videocard0″ to “Videocard1″
• Change the “Monitor” section as follows:

  Section "Monitor"
     Identifier        "Monitor0"
     HorizSync      30.0 - 100.0       #adjust to your monitor
     VertRefresh   50.0 - 94.0         #adjust to your monitor
     Option            "DPMS"
  EndSection
  

• Add a new “Monitor” section for the TV, just like the one before, but change “Monitor0″ to “Monitor1″
• Now we configure the possible resolutions for CRT and TV

  You will have to adjust them to your liking. “Screen0″ deals with the CRT, “Screen1″ with the TV.

  
  Section "Screen"
      Identifier     "Screen0"
      Device         "Videocard0"
      Monitor        "Monitor0"
      DefaultDepth    24
      Option         "metamodes" "CRT: 1600x1200 +0+0; CRT: 1400x1050 +0+0;
      CRT: 1280x1024 +0+0; CRT: 1024x768 +0+0; CRT: 800x600 +0+0;
      CRT: 640x480 +0+0"
      SubSection     "Display"
          Depth       24
          Modes      "1600x1200" "1280x1024" "1024x768" "800x600" "640x480"
      EndSubSection
  EndSection
  
  Section "Screen"
      Identifier     "Screen1"
      Device         "Videocard1"
      Monitor        "Monitor1"
      DefaultDepth    24
      Option         "metamodes" "TV: 1024x768 +0+0"
      SubSection     "Display"
          Depth       24
          Modes      "1600x1200" "1280x1024" "1024x768" "800x600" "640x480"
      EndSubSection
  EndSection
  

• Now you we are physically set up and can define a Serverlayout which defines how the monitors do correspond to each other.
  In this example the CRT will be the primary monitor whereas the TV can be reached by dragging the mouse cursor out the left side of your monitor.

  Section "ServerLayout"
      Identifier     "Layout0"
      Screen      0  "Screen0" 1024 0
      Screen      1  "Screen1" LeftOf "Screen0"
      InputDevice    "Keyboard0" "CoreKeyboard"
      InputDevice    "Mouse0" "CorePointer"
  EndSection
  

Now we are all done. Save xorg.conf and restart your Xserver(in Solaris logging out and in again will do the job).
A working copy of my file as an example can be downloaded here.

1. To do a quick test where the configuration won’t have to survive a reboot
2. On an interface in a global zone
3. On an interface in a non-global zone

These are the solutions we used:

1

$ ifconfig abrX:Y plumb

while “abrX” is the abbreviation for an installed interface.
Now you can use the new interface abrX:Y as you want.

2
We needed it to have an an external(Internet) and an internal address. So we made an entry in /etc/hosts for the primary address in the old fashioned way(we do not yet use NWAM):

10.5.250.100    fire

while we put the other addresses in /etc/hostname.abrX:

fire
addif 141.72.100.100/24

where /24 implies: netmask ffffff00 broadcast 141.72.100.255

3
We wanted the interfaces to be visible from the global zone via a local address while being available from the Internet. Therefore we defined a local address as in 2 via /etc/hosts and /etc/hostname.abrX, but we also included the Internet address in the zone configuration:

$ zonecfg -z zone1
zonecfg:zone1> select net physical=abrX
zonecfg:zone1:net> set address= 141.72.100.101/24
zonecfg:zone1:net> end
zonecfg:zone1> commit
zonecfg:zone1> exit

We will now take a look at IP instances, because using them we can achieve to only have the non-global zones visible in the Internet, while the global zone is only vulnerable in the local subnet.

The usage is pretty much straight forward – simply give the script
the needed information as to where the aliases file is located, the
account for MySQL and so forth.
Note: In the current implementation you have to create the database,
a user and a valid relay table yourself. If you don’t know how to do that
you can extract this information in my howto on migrating Postfix to
MySQL.

download

# Uncomment the following three lines to get this to compile on Solaris
 LDFLAGS=-lxnet
 CC=cc $(LDFLAGS) -DSOLARIS -DNO_FLOCK
 M="CC=$(CC)"
# These are currently unused, but will be needed again if we use flock() again
# CFLAGS=-I/usr/ucbinclude
# L="CC=$(CC) $(CFLAGS)"
# LDFLAGS=-L/usr/ucblib -lucb -lxnet
# end the Solaris section
# Non-Solaris version of "M"
#M="CC=$(CC) -O2 -Wall" DEBUG=-DNO_FLOCK
#V="VERSION=$(VERSION)"

# Debug
D="VERSION=$(VERSION)" DEBUG="-DDEBUG -DNO_FLOCK"

#FLAGS = -O2 -Wall
FLAGS = -g

afterwards a simple make did the job.

VIM / Cursor Keys
If you wonder why VIM keeps writing ‘A’, ‘B’, ‘C’, ‘D’ on your screen when it is supposed just to move the cursor in writing mode, the answer is that the cursor keys are not being mapped the right way. The solution is to extend your favourite .vimrc file with:

map! ^[OD ^[h
map! ^[OC ^[l
map! ^[OA ^[k
map! ^[OB ^[j

“/usr/ucb/cc: language optional software package not installed”
/usr/ucb/cc is only a wrapper to a real C compiler. Solaris 10 won’t have a C compiler preinstalled, normally. This means that you will have to install it yourself (Sun Studio would be a good idea, too). All you have to see to then is that your PATH is set correctly; meaning that /usr/ucb is either deleted or after your real compiler.

“WARNING: loghost could not be resolved.”
That’s because your syslogd doesn’t have a defined host to work on. To fix it simply add “loghost” to your 127.0.0.1 entry in the /etc/hosts file

127.0.0.1       localhost loghost

“Sendmail: My unqualified host name (domain) unknown;”
If you get this warning on computer startup, then you haven’t configured a fully qualified domain name for your system. If you don’t need a mailing system on your computer the solution is easy by simply disabling the service via:

svcadm disable sendmail

If you need a working mailsystem, there is a howto from SUN. ]]> http://blog.dispatched.ch/2008/10/04/first-installation-troubles-solaris/feed/ 0 http://blog.dispatched.ch/2008/10/04/amp-stack-in-solaris-0508/ http://blog.dispatched.ch/2008/10/04/amp-stack-in-solaris-0508/#comments Sat, 04 Oct 2008 16:18:27 +0000 Alain M. Lafon http://gefechtsdienst.de/?p=90 Most webservers do have a need for a decent AMP stack, but Solaris 05/08 doesn’t quite offer a solution out of the box, but if you know where to look you’re not far away from having a fine implementation.

• MySQL5I have made good experiences using MySQL5 from blastwave. Unfortunately it doesn’t run out of the box even tough it integrates nicely with SMF. When mysql is being called after installation you will see this error message:

  Warning: mysql(): Can't connect to local MySQL server through socket '/tmp/mysql.sock'

  The simple reason is that cswmysql5 couldn’t start properly, because there is no default config file. After copying the config file to the right location you can start the installation:

  $ cp /opt/csw/mysql5share/mysql/my-small.cnf /opt/csw/mysql5/my.cnf
  $ /opt/csw/mysql5/bin/mysql_install_db
  ...
  $ chown -R mysql:mysql /opt/csw/mysql5/var
  $ svcadm disable cswmysql5; svcadm enable cswmysql5

• Apache2

  $ pkg-get install apache2

• PHP

  $ pkg-get install php5

• Needed modules

  $ pkg-get install ap2_modphp5
  $ pkg-get install php5_gd
  $ pkg-get install
  php5_mysql

  Now all that’s left is to integrate PHP support into apache which is done by adding the following lines in your /opt/csw/apache2/etc/httpd.conf:

  LoadModule php5_module /opt/csw/apache2/libexec/libphp5.so
  <IfModule mod_php5.c>
    AddType application/x-httpd-php .php .phtml .php3
    AddType application/x-httpd-php-source .phps
  </IfModule>

  Now disable apache2, restart cswapache2 and you’re done.