Danish people speak HTML

The nice thing about Perl is the point, that there is always someone who has solved the problem you have at the moment. Yesterday i had the idea to put all articles in an addtional category based on their language as this is an bilingual blog. With a article count heading towards 6000, a manual process to do so, was out of question. Thus i´ve searched for an automatic way to do this. After googling for the problem i found the Lingua::Identify module. The usage of the module made the rest easy. The following script puts articles in a Serendipity-based blog in a category based on the English. 41 is the category ID for English articles, 42 is the category for articles in German. Interestingly articles with many HTML tags are frequently identified as written in Danish.


#!/usr/bin/perl

use Lingua::Identify qw(:language_identification);
use DBI;

my $dbh = DBI->connect( "DBI:mysql:databasename","username","password") || die "Database connection not made: $DBI::errstr";

my $sql = qq{SELECT id,body FROM serendipity_entries};
my $sth = $dbh->prepare($sql);
$sth->execute();
while(@ergebnis=$sth->fetchrow_array)
{
$id=$ergebnis[0];
$body =$ergebnis[1];
$lang =langof($body);
if (($lang ne "en") and ($lang ne "de")) {
next;
}
$cat=41 if $lang eq "en";
$cat=42 if $lang eq "de";
print "INSERT INTO serendipity_entrycat(entryid,categoryid) values($id,$cat);\n";
}

$sth->finish();
$dbh->disconnect();