I’m developing a web app right now and as some of you might know there is a very popular javascript command which is used for debugging. It’s invoked via console.log() and takes anything as an argument. If you use Firefox with Firebug or Safari 4 the value you pass to console.log will be printed and introspected via a debug window.
The problem is that I forget on a regular basis to remove those console.log statements. I wrote a very quick and dirty perl script which searches files with an .js extension for console.log. I put that script in the t/ folder and named it forbidden_words.t.
It works great so far and my test suite fails as soon as there is still a console.log around. I plan to extend this script to something like Test::ForbiddenWords or something where you can specify file extensions and strings (or regexes).
I couldn’t find anything like this on the CPAN, so give me a comment if you like the idea or know something better.
This script is in use for many years now and does a decent job. I set up a MySQL table which specifies the basic configuration options for an Apache2 virtual host:
CREATE TABLE `vhosts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`DocumentRoot` text NOT NULL,
`ServerName` varchar(255) NOT NULL DEFAULT '',
`ServerAlias` varchar(255) NOT NULL DEFAULT '',
`ScriptAlias` text NOT NULL,
`CustomLog` text NOT NULL,
PRIMARY KEY (`id`)
) ;
In /etc/apache2/conf.d/vhosts.perl.conf I’ve put this little script:
<Perl>
use DBI;
use File::Path;
my $dbh = DBI->connect( "DBI:mysql:vhosts", "apache", "password" );
my $sth = $dbh->prepare("select * from vhosts");
$sth->execute();
while ( my $vhost = $sth->fetchrow_hashref() ) {
unless ( -d ( split( / /, $vhost->{ScriptAlias} ) )[1] ) {
mkpath( ( split( / /, $vhost->{ScriptAlias} ) )[1] );
}
unless ( -d $vhost->{DocumentRoot} ) {
mkpath( $vhost->{DocumentRoot} );
}
unless ( -e ( split( / /, $vhost->{CustomLog} ) )[0] ) {
my $dir = $vhost->{CustomLog};
$dir =~ s/(.*)\/.*?$/$1/;
mkpath($dir);
}
undef( $vhost->{id} );
push( @VirtualHost, { "*" => $vhost } );
}
$sth->finish();
$dbh->disconnect;
</Perl>As you can see the script uses all columns from the table to set up the virtual host. You can easily add more columns (e.g. ErrorLog) to the table as long as their name is known to the apache2 configuration.
After you added a new vhost to the database you need to restart the apache server so that the perl script is run and the vhost is set up.
This will create the directories of any vhost which has been added to the table “vhosts” and create a virtual host inside of apache. This is very handy if you need to maintain multiple sites which share the same layout and need basic functionality.
I managed to pack my work on code completion to a distribution and released it to the CPAN. The source code is managed on github. Use it, watch it, fork it, break it, fix it, patch it … you know the game!
Watch the demo for new features (no audio).
So what is missing?
It would be great if code completion works for chained method calls:
# DateTime
my $dt = DateTime->new(year => 2009);
$dt->add( months => 1)->_ # we are still a DateTime object!
# in a DBIC environment
my $rs = $schema->resultset('Foo')->_ # we get a MySchema::ResultSet::FooWhat we need is a way to define the return values of methods. Moose’s type system is not much of a help because you cannot introspect what class is behind a custom type.
I was thinking about an IntelliPerl profile which is stored in the home directory as well as in each project. Those two are merged and define method signatures and variable types.
Possible syntax:
# ~/.intelliperl
$schema isa DBIx::Class::Schema;
$rs isa DBIx::Class::ResultSet;
$dt isa DateTime;
method DBIx::Class::ResultSet::search returns DBIx::Class::ResultSet
# ~/workspace/MyApp/.intelliperl
$schema isa MyApp::Model::DBIC
$rs isa MyApp::Schema::ResultSet
A different approach would be to add the method signature to the POD:
=head2 mymethod
=for intelliperl
method mymethod ($dt DateTime) returns DateTime
This will make $dt inside mymethod a DateTime variable and the returned value is a DateTime object as well.
For now only TextMate is supported. See the documentation for details.
I’d love to see more Devel::IntelliPerl::Editor::s!
It is pretty hard to write a code completion script for perl since it’s hard to find out of which type a variable is. I use simple regexes and a comment to achieve this. Here is how it looks like:
It consists of a perl script which reads the current file from STDIN and gets the current line number as well as the cursor position in that line. The script is run if the cursor is behind a -> and the string before that looks like a class or a variable. If it’s a variable the script travels the lines up until it finds something like $var = Class->new, $var = new Class or # $var isa Class.
Class::MOP::Class loads that class and retrieves all method names. It also evaluates the prefix which has been entered behind the -> and displays those methods only which have the same prefix. Private methods are moved to the buttom as well as capitalized method names.
The second part of this script is embedded in TextMate. TextMate allows to define custom commands. You can define how the data shoud be rendered depending on the return value of the script. In this case STDERR is printed directly into the editor (if there is only one method left) and STDOUT is printed as tool tip.
I’d like to hear what you think about it and whether there are better / other approaches. I should probably use PPI instead of regexes to parse the document. This is one of the reasons why I did not release any code yet. Another shortcoming is speed. Especially for large classes like Catalyst or DateTime it takes a noticeable time until you get the results.
So long… I’ll keep you posted.
TextMate has a number of great bundles which help you develop in many many languages. The feature I miss most is formatting for JavaScript just like Perl Tidy does it for Perl.
I recently saw JavaScript::Beautifier in the “Recent” list on the CPAN and was wondering if I can bind it to TextMate. This is how I did it:
First of all, install it:
sudo cpan -i JavaScript::Beautifier
Next, add the command to TextMate. Open the Bundle Editor (Bundles/Bundle Editor/Show Bundle Editor).
Then select JavaScript and click on “New Command” (bottom left).
Name it “Beautifier” (or whatever you want) and paste the following in the command text field:
require_cmd js_beautify.pl 'sudo cpan -i JavaScript::Beautifier'
js_beautify.pl $TM_FILEPATH
Set the rest of the options like this:
Now you are ready to go! Open up a JavaScript document and hit Shift+Crtl+H and the current document should be formatted.
I recently ran across the probem that there are many people with almost identical last names and it is hard to guess every possible spelling until you finally find the person you were looking for.
In german, there are a lot of possible spellings for the name “Maier” which all sound the same: Meyer, Meier, Mayer, Mayr, Meyr etc.
A phonetic algorithm reduces a given word to a digest which is the same for all names, which sound similar. After asking for a proper name on #dbix-class I came up with the DBIx-Class-PhoneticSearch module.
For now it is only avaiable from github. But sometime soon it will be on the CPAN :-)
The usage is pretty easy:
package MySchema::User;
use base 'DBIx::Class';
__PACKAGE__->load_components(qw(PhoneticSearch Core));
__PACKAGE__->table('user');
__PACKAGE__->add_columns(
id => { data_type => 'integer', auto_increment => 1, },
surname => { data_type => 'character varying',
phonetic_search => 1 },
forename => { data_type => 'character varying',
phonetic_search => { algorithm => 'Koeln',
no_indices => 1 } },
);
__PACKAGE__->set_primary_key('id');
__PACKAGE__->resultset_class('DBIx::Class::ResultSet::PhoneticSearch');This defines a result class with a forename and surname column. Both are phonetic-enabled. forename uses the Koeln algorithm, which has been optimized for german names and words. Make sure you deploy() that schema again or add the two columns to your schema.
ALTER TABLE `user` ADD COLUMN `surname_phonetic_phonix` CHARACTER VARYING;
ALTER TABLE `user` ADD COLUMN `forename_phonetic_koeln` CHARACTER VARYING;
Now you can search for any user by a similar sounding name:
$rs = $schema->resultset('User');
$rs->create({ forename => 'John', surname => 'Night' });
$rs->search_phonetic({ forename => 'Jon' })->first->forename; # John
$rs->search_phonetic({ surname => 'Knight' })->first->surname; # NightThe default algorithm is Phonix which is IMHO far superior to the popular Soundex algorithm. E. g. the last example (Knight -> Night) does not work with Soundex.
Currently I use plain SQL statements to create reports which, for instance, show me the number of logins of all users on a 5 minutes scale.
One of those statements looks similar to this:
INSERT INTO report_item
(
SELECT '2009-05-23 02:30:00+0200'::timestamp WITH time zone + (s.a || ' minute')::interval AS timestamp,
COUNT(source."created_on") AS value
FROM generate_series(0,595,5) AS s(a)
LEFT JOIN
(SELECT "created_on" FROM "session") AS source
ON source."created_on" >= '2009-05-23 02:30:00+0200'::timestamp WITH time zone + (s.a || ' minute')::interval
AND source."created_on" < '2009-05-23 02:30:00+0200'::timestamp WITH time zone + (s.a + 5 || ' minute')::interval
GROUP BY a
);This gives me the number of logins (or new sessions) per 5 minute time slot. Now it’s pretty easy to read the data from report_item and create a graph or do other nice things with it.
I was thinking about a more generic way to do these reports so that the user can choose, what kind of report he wants.
So I started to create a result class Report with the following columns:
- group_by
The column which is used to group rows together. This is always a timestamp column, e.g. created_on
- interval_type
Group by minutes, hours, days, weeks etc. (defaults to minutes)
- interval_length
Group by that amount of interval_type (defaults to 5)
- aggregate_by
Which SQL function should be used to aggregate data in a time slot (defaults to COUNT)
- aggregate
Which column should be aggregated
- query
Which data should be aggregated? This accepts a DBIC result set or a plain SQL statement
After you created a report in that table you call $report->create_report on that row. This will execute the query above with the correct values filled in and the result will be stored in a table result_item, which has a foreign key report_id to the report.
It does a decent job so far and I was wondering if anyone else did something like this before?
If anyone is interested in the code I can make a dist and upload it to github. Since the SQL is Postgres only I don’t want to push it to the CPAN.
I was wondering how you design your base classes for DBIC result(set) classes. I think it’s a good idea both in terms of startup speed and ease of use to not use DBIx::Class as base of your result classes but use a custom one instead. The same applies for the resultset classes. Here is how I designed my DBIC classes:
Make sure you use load_namespacesin MySchema.pm!
Directory structure:
MySchema.pm
MySchema/
Base/
Result.pm
ResultSet.pm
Result/
User.pm
ResultSet/
User.pm
This has the advantage that the base classes in base are not being loaded. All classes in Result use MySchema::Base::Result as their base class. Classes in ResultSet use MySchema::Base::ResultSet respectively.
My current result base class looks like this:
package MySchema::Base::Result;
use strict;
use warnings;
use base 'DBIx::Class';
__PACKAGE__->load_components(qw(RandomColumns InflateColumn::FS TimeStamp Core));
__PACKAGE__->table('dummy');
__PACKAGE__->add_columns(
id => {
data_type => 'character',
is_random => {size => 10, set => ['a'..'z','A'..'Z'], check => 1},
size => 10,
},
created_on => {
data_type => 'timestamp with time zone',
set_on_create => 1,
is_nullable => 1
},
updated_on => {
data_type => 'timestamp with time zone',
set_on_create => 1,
set_on_update => 1,
is_nullable => 1
});
__PACKAGE__->set_primary_key('id');
sub sqlt_deploy_hook {
my ($self, $sqlt_table) = @_;
map { $sqlt_table->drop_constraint($_) if $_->type eq "FOREIGN KEY" }
$sqlt_table->get_constraints;
}
1;So what is going on here?
First I use load_components to load all the components which are valuable to all of my result classes. Since I always use a random combination of characters for the primary key column id I load RandomColumns. InflateColumn::FS makes sure that any BLOB is stored on the file system rather than in the database. TimeStamp can set columns to the current time on either update or create. It also loads InflateColumn::DateTime which is used to inflate any date or time columns to a DateTime object.
You need to define a table name, otherwise DBIC fails to compile. This table will never be created or seen in any of your result classes as long as you overwrite it in each class. So just name it dummy or something and you are good.
Next I create a couple of columns which should be avaiable on all result classes. My primary key is always called id so create that column for all my classes. To use the RandomColumns component you have to add that is_random line to the column definition. This will create a random 10-byte long string. Since I’m a little bit paranoid about collision I set the check parameter so the component checks before inserting a new row if a column with that id is already there. Although this is very unlikely because there are 52^10 combinations, quite a lot…
The created_on and updated_on columns use the features of TimeStamp to set the time when the record was created or updated respectively. timestamp with time zone is a PostgreSQL specific column. You might need to the correct data type of your dbms.
I use SQL::Translator, DBIx::Class::Schema::Versioned and a few lines of code to deploy and update my database schema. Foreign key constraints on the database slow things down and DBIC handles them anyway so I decided to drop them from the SQL::Translator output. this is done in the sqlt_deploy_hook method.
Growl is a nice notification utility for Mac OS X. There are many applications or plugins avaiable which interact with Growl (e. g. Skype, Mail, VLC etc.).
I thought it would be nice if I could redirect the catalyst debug output to it as well. Log::Dispatch has a nice appender Log::Dispatch::MacGrowl. Load Log::Dispatch as plugin in MyApp.pm (after ConfigLoader):
use Catalyst qw/-Debug
ConfigLoader
Log::Dispatch
Static::Simple/;and configure your app accordingly:
<Log::Dispatch>
class = MacGrowl
name = growl
app_name = MyAppGrowls
title = MyApp
sticky = 0
min_level = notice
format = [%p] %m %n
</Log::Dispatch>
I cannot recommend to set min_level to info or below because this will add all start up log statements to be rendered on your screen. Kind of annoying.
Put a log statement anywhere in your code and start up your dev server. A growl notification should appear when you hit the log statement.
You can get a sample application MyAppGrowls from GitHub.
Make sure you have the following modules installed (or run perl Makefile.PL):
- Catalyst::Plugin::Log::Dispatch
- Log::Dispatch::Config
- Log::Dispatch::MacGrowl
Writing documentation is sometimes hard but inevitable. If you write code which is published on the CPAN you can always access the documentation through search.cpan.org. For those cases where you cannot publish your code on the CPAN it would be great if you had a web site like CPAN where you can access the documentation. For this reason I wrote Pod::Browser a while back. Pod::Browser is a browser for all your local modules and all the modules from the CPAN. The interface is rendered by ExtJS, a powerful JavaScript framework.
After installing Pod::Browser and running the server (pod_browser_server.pl) you should be able to access the browser via http://localhost:3000. On the right hand side of the page is a tree which contains all the modules installed on your system. The main page contains a search box where you can search the CPAN via it’s XML interface. If you open a module which is also installed on your system, the local pod will be served. Otherwise the pod is read from the CPAN. Each module has its own ExtJS tab. This allows to have multiple documents open at the same time in a single browser window. The table of contents is shown in the upper left panel.
Since this browser has been implemented as a single Catalyst controller, you can simply create an empty controller in your application and use Catalyst::Controller::POD as base class:
package MyApp::Controller::Docs;
use strict;
use warnings;
use base 'Catalyst::Controller::POD';
__PACKAGE__->config(
inc => 1,
namespaces => [qw(Catalyst::Manual*)],
self => 1,
);
1;This will make the documentation to your application avaiable from /docs. The pod tree will also contain all the modules which are listed in the namespaces config attribute (in this case Catalyst::Manual).
For more information on Pod::Browser and Catalyst::Controller::POD please read the documentation on the CPAN.
Warning:
If you set inc to 1 and leave the namespaces parameter blank it can take a while until all modules are indexed and presented in your browser. Be either patient or restrict the namespace to a sane subset of your modules.
1
Back to top