Invalid UTF-8 for Postgres, Perl thinks they're ok

Posted by gorilla on Stack Overflow See other posts from Stack Overflow or by gorilla
Published on 2010-04-16T22:20:18Z Indexed on 2010/04/16 22:23 UTC
Read the original article Hit count: 531

Filed under:
|
|
|

I'm running perl 5.10.0 and Postgres 8.4.3, and strings into a database, which is behind a DBIx::Class.

These strings should be in UTF-8, and therefore my database is running in UTF-8. Unfortunatly some of these strings are bad, containing malformed UTF-8, so when I run it I'm getting an exception

DBI Exception: DBD::Pg::st execute failed: ERROR: invalid byte sequence for encoding "UTF8": 0xb5

I thought that I could simply ignore the invalid ones, and worry about the malformed UTF-8 later, so using this code, it should flag & ignore the bad titles.

if(not utf8::valid($title)){
   $title="Invalid UTF-8";
}
$data->title($title);
$data->update();

However perl seems to think that the strings are valid, but it still throws the exceptions.

How can I get perl to detect the bad UTF-8?

© Stack Overflow or respective owner

Related posts about perl

Related posts about postgresql