Archive for September, 2007

Unit Tests for HTML Validation

September 29th, 2007

It is now very easy to integrate testing for valid HTML into your web application’s test environment using the W3C’s validator and PHPUnit. I just committed a sample unit test for PHPUnit which will checks your html markup.

The test cases or suites are very easy to integrate into your existing environment with the Services_W3C_HTMLValidator package.

First you’ll need the PEAR package for html validation:
pear install Services_W3C_HTMLValidator-beta

And then you should copy down the two sample PHPUnit files ValidationSuite.php and URIValidationTest.php.

Within the ValidationSuite.php file, modify the array of URLs to check and simply run the tests with php ValidationSuite.php

A validation test will be automatically added for each URI you wish to validate.

– NOTE: If you plan on validating many pages, consider installing a local copy of the W3C validator – which isn’t that complicated and you’ll get faster results.

RoR has a similar plugin for html validation called ResponsibleMarkup, and I think the title is appropriately named.

Happy validating!

Settling In To The New Garage

September 28th, 2007

We bought our new house just about a year ago. And in that time I’ve been settling in to the new garage – extra deep. Which has given me some more space for my woodworking tools and general organization. While still having room for cars.

The first step was some simple cabinets:
lighting installed
I used white laminated paneling for the faces of the doors which makes a handy place for labeling the contents.
more cabinet labels

I also installed some cheap florescent lighting right in front of the cabinets – giving me plenty of light for working, not for photography though.

The next step was some pegboard for organizing those frequently used hand tools to clear off my workbench… as you can see I’ve got a ways to go before the bench is completely clean.

pegboard & workbench

There have been some other woodworking projects I’ve worked on recently – mainly a wood privacy fence from reclaimed wood. Which looks great… and keeps the dog in.

iTunes U Authentication Class

September 21st, 2007

Aaron Axelsen’s iTunes U autentication class written in PHP is now installable through UNL’s local PEAR channel.

pear channel-discover pear.unl.edu
pear install unl/UNL_iTunesU-beta

The class is nearly identical except the constructor now accepts an associative array of options to set for the class, and added support for PEAR packages if PHP doesn’t have curl or hash extensions.

Our university is just getting started with iTunes U and authentication is the first step – big thanks to Aaron @ UWW for porting Apple’s perl connection script to PHP.

MythTV & Ubuntu

September 20th, 2007

GRRR. ARGH! AVAST Ye SALTY UBUNTU!
Sorry, ‘talk like a pirate day’ has passed, today is ‘frustrated with Ubuntu + MythTV day’ — which is almost synonymous.

If you get this error:

bbieber@baddog:~/mythtv/mythtv$ /usr/local/bin/mythfrontend
/usr/local/bin/mythfrontend: error while loading shared libraries: libmythtv-0.20.so.0: cannot open shared object file: No such file or directory

Pretty simple to fix, create a configuration file for ld.so which tells the linker where to find your libs -
sudo nano /etc/ld.so.conf.d/mythtv.conf

Then add this one line to it:

/usr/local/lib

and run sudo ldconfig

Then you should be good to go.

I now have MythTV running the latest from SVN on Ubuntu, only now my Gentoo box has an older version.
Take my advice, if you ever build a mythtv setup with multiple frontends – use the same OS on all your machines.
My setup consists of one custom built 3.4GHz P4 HTPC in our living room with 2 Happauge PVR 150 capture cards (dual tuners), a 1.2GHz Celeron Compaq Evo frontend w/Streamzap IR in the bedroom, and my desktop which is a 2.8GHz P4 with a pcHDTV HD-5500.
Eh… the best way to visualize this is to flex my Open Office Draw skillz, er maybe not.
On my master backend I went with Gentoo because I was always modifying and compiling the drivers for my PVR 150 capture cards (evidently I was the second guy after Chris Kennedy to use the PVR150 in Linux)… only now Ubuntu lags behind so far, I had to switch from using the packages to SVN. Now of course, Ubuntu is using protocol version 36, and Gentoo is behind with version 35. GRR. I can never win.

Why Aren’t You Using APC?

September 20th, 2007

I got a new Mac this week – Quad Core Dual 2.66 Intel, pretty sweet. Anyways, the Apple Migration Assistant can only transfer so much. I had to setup PHP 5, MySQL, as well as quite a few other manual install and custom configuration files. Well, I wasn’t getting the performance I was expecting out of my PHP apps, until I remembered that APC wasn’t installed.

Now I’m not that big of an optimization freak, so my code isn’t that lean and mean, but man APC does some amazing stuff.
Here’s the results of an apachebench [ab] on my localhost -

Without APC (~40 req/s):

ucommbieber:~ bbieber$ ab -k -c 50 -t 300 http://localhost/events/
This is ApacheBench, Version 1.3d <$Revision: 1.73 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Finished 12202 requests
Server Software: Apache/1.3.33
Server Hostname: localhost
Server Port: 80

Document Path: /events/
Document Length: 23809 bytes

Concurrency Level: 50
Time taken for tests: 300.002 seconds
Complete requests: 12202
Failed requests: 952
(Connect: 0, Length: 952, Exceptions: 0)
Broken pipe errors: 0
Non-2xx responses: 2
Keep-Alive requests: 0
Total transferred: 269977940 bytes
HTML transferred: 267940001 bytes
Requests per second: 40.67 [#/sec] (mean)
Time per request: 1229.31 [ms] (mean)
Time per request: 24.59 [ms] (mean, across all concurrent requests)
Transfer rate: 899.92 [Kbytes/sec] received

Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 4.0 0 76
Processing: 117 1226 2243.5 1065 33964
Waiting: 117 1226 2243.4 1065 33936
Total: 117 1227 2246.6 1065 33990

Percentage of the requests served within a certain time (ms)
50% 1065
66% 1097
75% 1119
80% 1133
90% 1184
95% 1375
98% 1764
99% 3704
100% 33990 (last request)

With APC (~208 req/s)

ucommbieber:~ bbieber$ ab -k -c 50 -t 300 http://localhost/events/
This is ApacheBench, Version 1.3d <$Revision: 1.73 $> apache-1.3
Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Finished 50000 requests
Server Software: Apache/1.3.33
Server Hostname: localhost
Server Port: 80

Document Path: /events/
Document Length: 23809 bytes

Concurrency Level: 50
Time taken for tests: 239.538 seconds
Complete requests: 50000
Failed requests: 0
Broken pipe errors: 0
Keep-Alive requests: 0
Total transferred: 1198823976 bytes
HTML transferred: 1190473809 bytes
Requests per second: 208.74 [#/sec] (mean)
Time per request: 239.54 [ms] (mean)
Time per request: 4.79 [ms] (mean, across all concurrent requests)
Transfer rate: 5004.73 [Kbytes/sec] received

Connnection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 2.0 0 101
Processing: 19 239 43.4 234 1489
Waiting: 19 238 43.4 233 1489
Total: 19 239 44.1 234 1489

Percentage of the requests served within a certain time (ms)
50% 234
66% 238
75% 241
80% 244
90% 254
95% 265
98% 294
99% 349
100% 1489 (last request)

And all you’ve gotta do is:
sudo pear install pecl/apc
sudo apachectl graceful

I went from 40 requests per second to 208 requests per second. Wowzers. You should seriously install it.

Best Practices for Distributive Web Designs

September 19th, 2007

Distributive designs are very important in large institutions – especially educational environments. Through two major design changes in 4 years, and a distributed design over a thousand of virtual hosts and 50+ servers I’ve learned a few things about creating a distributive web design which I thought I could share with the community. The challenges are many when distributing a web design which needs to work across many web servers and workstations utilizing different operating systems, but by following a few simple techniques you can simplify design upgrades and synchronization for your distributive web design.

Distributive designs are becoming more popular even as CSS combined with web standards takes hold within educational institutions. By abstracting the presentation from the content of your site, the visual appearance can be applied externally for small or drastic design changes. Through the use of Cascading Style Sheets (CSS), designers can make fluid design changes across an entire website – as long as this stylesheet files (and perhaps other files) are synchronized between all the servers.
The challenge is how to link to this common style. There are many methods of addressing files locally on the same server or externally on another server. I’ll step through the various linking methods and highlight some advantages and disadvantages of each.

“Why not use absolute addressing to one official copy of the CSS file?”
Such as: <link rel=”stylesheet” href=”http://www.example.com/officialsitestyle.css” />

This practice works well for small designs and close teams.

Advantages:

  • Very simple to get running
  • Easy to update
  • Cached once

Disadvantages:

  • Large designs will require more than just a CSS file (javascript, snippets of html etc)
  • Creates notices of insecure items when accessed through https
  • Some IDE’s will not render a design if the CSS is not locally referenced

“What about relative addressing?”
Such as: <link rel=”stylesheet” href=”officialsitestyle.css” />

Relative addressing works well for small sites all working at the same level.
Advantages:

  • Simple to distribute
  • Works in nearly all IDE’s
  • Files can be previewed offline using file://

Disadvantages:

  • Limited cache support
  • Hard to synchronize many copies without symbolic links/aliases all over
  • Leads to many duplicate copies on sites with a large vertical hierarchy

“What about site – root addressing?”
Such as: <link rel=”stylesheet” href=”/officialsitestyle.css” />
Advantages:

  • Moderately cacheable (domain specific caching)
  • Easy to distribute
  • Each server only needs to maintain one copy

Disadvantages:

  • Files can’t be previewed directly using file:// as the browser has no context of the site root

As you can see, each method has advantages and disadvantages. If you have a very small design style, and don’t have to worry about secure pages or can easily modify the css link to be secure, you should be fine with absolute linking to one file. Relative addressing works for small teams where communication is good, and you aren’t developing within a deep site hierarchy. The disadvantage is that it leads to many copies which doesn’t work well for pushing out updates.

In our workshop, we use site-root relative addressing, meaning the files are all referenced from the site root so each server just maintains one copy. To overcome the disadvantage I noted, Dreamweaver 9 can generate a temporary file with the content of the referenced CSS files – which fixes that issue.

Site-root addressing works the best for our distributive web design, it may not work for you, but I hope this gives users some advice about the differences when they choose a path to head down. Keep in mind the technical challenges related to updates – keeping many servers in sync can be a difficult task. Expect a post on synchronization when I get around to it.

Installing the Validator from CVS on Ubuntu

September 4th, 2007

Here are some notes on installing the validator, so you can test and work with the CVS version on Ubuntu Linux.

On my machine I use Eclipse which stores files in ~/workspace/ – all of my paths are based off of this, and I add an alias to apache so that all my eclipse projects are available at http://localhost/workspace/{project}

For my setup, I have checked out the validator source code in ~/workspace/validator/ which shows up at http://localhost/workspace/validator/. Modify the paths if your CVS checkout is in a different location.

To get a snapshot of the latest files for the validator you can use:
cvs -d :pserver:anonymous:anonymous@dev.w3.org:/sources/public checkout validator, which will create a validator directory with the latest files.

Make sure your apache configuration has includes enabled, and will process the cgi script — modify the /etc/apache2/apache2.conf file — sudo nano /etc/apache2/apache2.conf, and add the following:

# This is the directory where you have the validator's "check"
# script as well as its and *.html, *.css etc files.

<Directory /home/bbieber/workspace/validator/htdocs>
Options ExecCGI IncludesNOEXEC Indexes MultiViews FollowSymlinks
AllowOverride None
AddHandler server-parsed .html
AddCharset utf-8 .html
</Directory>

# Tell httpd that "check" is a CGI script.

<Location "/workspace/validator/htdocs/check">
SetHandler cgi-script
</Location>

Then, in the CVS files you have checked out, symbolically link the ‘check’ file within the htdocs directory:

cd htdocs
ln -s ../httpd/cgi-bin/check check

So that the file check -> ../httpd/cgi-bin/check


cd ../httpd
chmod +x check

Install the modules and dependencies:

sudo apt-get install opensp xmlto
sudo cpan install Config::General Encode::HanExtra Encode::JIS2K HTML::Encoding HTML::Template Test::Exception Class::Accessor Test::Builder Sub::Uplevel Test::Builder::Tester
wget http://superb-west.dl.sourceforge.net/sourceforge/openjade/OpenSP-1.5.2.tar.gz
gunzip OpenSP-1.5.2.tar.gz
tar -xvf OpenSP-1.5.2.tar
cd OpenSP-1.5.2
./configure
make
sudo make install
sudo cpan install SGML::Parser::OpenSP XML::LibXML Net::IP