Some time back I created a data base driven ASP.NET Resource Provider along with some tools that make it easy to edit ASP.NET resources interactively in a Web application. One of the small helper features of the interactive resource admin tool is the ability to do simple translations using both Google Translate and Babelfish. Here's what this looks like in the resource administration form: When a resource is displayed, the user can click a Translate button and it will show the current resource text and then lets you set the source and target languages to translate. The Go button fires the translation for both Google and Babelfish and displays them - pressing use then changes the language of the resource to the target language and sets the resource value to the newly translated value. It's a nice and quick way to get a quick translation going. Ch… Ch… Changes Originally, both implementations basically did some screen scraping of the interactive Web sites and retrieved translated text out of result HTML. Screen scraping is always kind of an iffy proposition as content can be changed easily, but surprisingly that code worked for many years without fail. Recently however, Google at least changed their input pages to use AJAX callbacks and the page updates no longer worked the same way. End result: The Google translate code was broken. Now, Google does have an official API that you can access, but the API is being deprecated and you actually need to have an API key. Since I have public samples that people can download the API key is an issue if I want people to have the samples work out of the box - the only way I could even do this is by sharing my API key (not allowed). However, after a bit of spelunking and playing around with the public site however I found that Google's interactive translate page actually makes callbacks using plain public access without an API key. By intercepting some of those AJAX calls and calling them directly from code I was able to get translation back up and working with minimal fuss, by parsing out the
JSON these AJAX calls return. I don't think this particular Warning: This is hacky code, but after a fair bit of testing I found this to work very well with all sorts of languages and accented and escaped text etc. as long as you stick to small blocks of translated text. I thought I'd share it in case anybody else had been relying on a screen scraping mechanism like I did and needed a non-API based replacement. Here's the code: /// <summary>
/// Translates a string into another language using Google's translate API
JSON calls.
/// <seealso>Class TranslationServices</seealso>
/// </summary>
/// <param name="Text">Text to translate. Should be a single word or sentence.</param>
/// <param name="FromCulture">
/// Two letter culture (en of en-us, fr of fr-ca, de of de-ch)
/// </param>
/// <param name="ToCulture">
/// Two letter culture (as for FromCulture)
/// </param>
public string TranslateGoogle(string text, string fromCulture, string toCulture)
{
fromCulture = fromCulture.ToLower();
toCulture = toCulture.ToLower();
// normalize the culture in case something like en-us was passed
// retrieve only en since Google doesn't support sub-locales
string[] tokens = fromCulture.Split('-');
if (tokens.Length > 1)
fromCulture = tokens[0];
// normalize ToCulture
tokens = toCulture.Split('-');
if (tokens.Length > 1)
toCulture = tokens[0];
string url = string.Format(@"http://translate.google.com/translate_a/t?client=j&text={0}&hl=en&sl={1}&tl={2}",
HttpUtility.UrlEncode(text),fromCulture,toCulture);
// Retrieve Translation with HTTP GET call
string html = null;
try
{
WebClient web = new WebClient();
// MUST add a known browser user agent or else response encoding doen't return UTF-8 (WTF Google?)
web.Headers.Add(HttpRequestHeader.UserAgent, "Mozilla/5.0");
web.Headers.Add(HttpRequestHeader.AcceptCharset, "UTF-8");
// Make sure we have response encoding to UTF-8
web.Encoding = Encoding.UTF8;
html = web.DownloadString(url);
}
catch (Exception ex)
{
this.ErrorMessage = Westwind.Globalization.Resources.Resources.ConnectionFailed + ": " +
ex.GetBaseException().Message;
return null;
}
// Extract out trans":"...[Extracted]...","from the
JSON string
string result = Regex.Match(html, "trans\":(\".*?\"),\"", RegexOptions.IgnoreCase).Groups[1].Value;
if (string.IsNullOrEmpty(result))
{
this.ErrorMessage = Westwind.Globalization.Resources.Resources.InvalidSearchResult;
return null;
}
//return WebUtils.DecodeJsString(result);
// Result is a JavaScript string so we need to deserialize it properly
JavaScriptSerializer ser = new JavaScriptSerializer();
return ser.Deserialize(result, typeof(string)) as string;
}
To use the code is straightforward enough - simply provide a string to translate and a pair of two letter source and target languages:
string result = service.TranslateGoogle("Life is great and one is spoiled when it goes on and on and on", "en", "de");
TestContext.WriteLine(result);
How it works
The code to translate is fairly straightforward. It basically uses the URL I snagged from the Google Translate Web Page slightly changed to return a
JSON result (&client=j) instead of the funky nested PHP style
JSON array that the default returns.
The
JSON result returned looks like this:
{"sentences":[{"trans":"Das Leben ist großartig und man wird verwöhnt, wenn es weiter und weiter und weiter geht","orig":"Life is great and one is spoiled when it goes on and on and on","translit":"","src_translit":""}],"src":"en","server_time":24}
I use WebClient to make an HTTP GET call to retrieve the
JSON data and strip out part of the full
JSON response that contains the actual translated text. Since this is a
JSON response I need to deserialize the
JSON string in case it's encoded (for upper/lower ASCII chars or quotes etc.).
Couple of odd things to note in this code:
First note that a valid user agent string must be passed (or at least one starting with a common browser identification - I use Mozilla/5.0). Without this Google doesn't encode the result with UTF-8, but instead uses a ISO encoding that .NET can't easily decode. Google seems to ignore the character set header and use the user agent instead which is - odd to say the least.
The other is that the code returns a full
JSON response. Rather than use the full response and decode it into a custom type that matches Google's result object, I just strip out the translated text. Yeah I know that's hacky but avoids an extra type and firing up the JavaScript deserializer. My internal version uses a small DecodeJsString() method to decode Javascript without the overhead of a full
JSON parser.
It's obviously not rocket science but as mentioned above what's nice about it is that it works without an Google API key. I can't vouch on how many translates you can do before there are cut offs but in my limited testing running a few stress tests on a Web server under load I didn't run into any problems.
Limitations
There are some restrictions with this: It only works on single words or single sentences - multiple sentences (delimited by .) are cut off at the
".". There is also a length limitation which appears to happen at around 220 characters or so. While that may not sound like much for typical word or phrase translations this this is plenty of length.
Use with a grain of salt - Google seems to be trying to limit their exposure to usage of the Translate APIs so this code might break in the future, but for now at least it works.
FWIW, I also found that Google's translation is not as good as Babelfish, especially for contextual content like sentences. Google is faster, but Babelfish tends to give better translations. This is why in my translation tool I show both Google and Babelfish values retrieved. You can check out the code for this in the West Wind West Wind Web Toolkit's TranslationService.cs file which contains both the Google and Babelfish translation code pieces. Ironically the Babelfish code has been working forever using screen scraping and continues to work just fine today. I think it's a good idea to have multiple translation providers in case one is down or changes its format, hence the dual display in my translation form above.
I hope this has been helpful to some of you - I've actually had many small uses for this code in a number of applications and it's sweet to have a simple routine that performs these operations for me easily.
Resources
Live Localization Sample
Localization Resource Provider Administration form that includes options to translate text using Google and Babelfish interactively.
TranslationService.cs
The full source code in the West Wind West Wind Web Toolkit's Globalization library that contains the translation code.
© Rick Strahl, West Wind Technologies, 2005-2011Posted in CSharp HTTP
Tweet
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
