Search Results

Search found 175 results on 7 pages for 'niels bom'.

Page 1/7 | 1 2 3 4 5 6 7  | Next Page >

  • UTF-8 HTML and CSS files with BOM (and how to remove the BOM with Python)

    - by Cameron
    First, some background: I'm developing a web application using Python. All of my (text) files are currently stored in UTF-8 with the BOM. This includes all my HTML templates and CSS files. These resources are stored as binary data (BOM and all) in my DB. When I retrieve the templates from the DB, I decode them using template.decode('utf-8'). When the HTML arrives in the browser, the BOM is present at the beginning of the HTTP response body. This generates a very interesting error in Chrome: Extra <html> encountered. Migrating attributes back to the original <html> element and ignoring the tag. Chrome seems to generate an <html> tag automatically when it sees the BOM and mistakes it for content, making the real <html> tag an error. So, using Python, what is the best way to remove the BOM from my UTF-8 encoded templates (if it exists -- I can't guarantee this in the future)? For other text-based files like CSS, will major browsers correctly interpret (or ignore) the BOM? They are being sent as plain binary data without .decode('utf-8'). Note: I am using Python 2.5. Thanks!

    Read the article

  • Variant Management– Which Approach fits for my Product?

    - by C. Chadwick
    Jürgen Kunz – Director Product Development – Oracle ORACLE Deutschland B.V. & Co. KG Introduction In a difficult economic environment, it is important for companies to understand the customer requirements in detail and to address them in their products. Customer specific products, however, usually cause increased costs. Variant management helps to find the best combination of standard components and custom components which balances customer’s product requirements and product costs. Depending on the type of product, different approaches to variant management will be applied. For example the automotive product “car” or electronic/high-tech products like a “computer”, with a pre-defined set of options to be combined in the individual configuration (so called “Assembled to Order” products), require a different approach to products in heavy machinery, which are (at least partially) engineered in a customer specific way (so-called “Engineered-to Order” products). This article discusses different approaches to variant management. Starting with the simple Bill of Material (BOM), this article presents three different approaches to variant management, which are provided by Agile PLM. Single level BOM and Variant BOM The single level BOM is the basic form of the BOM. The product structure is defined using assemblies and single parts. A particular product is thus represented by a fixed product structure. As soon as you have to manage product variants, the single level BOM is no longer sufficient. A variant BOM will be needed to manage product variants. The variant BOM is sometimes referred to as 150% BOM, since a variant BOM contains more parts and assemblies than actually needed to assemble the (final) product – just 150% of the parts You can evolve the variant BOM from the single level BOM by replacing single nodes with a placeholder node. The placeholder in this case represents the possible variants of a part or assembly. Product structure nodes, which are part of any product, are so-called “Must-Have” parts. “Optional” parts can be omitted in the final product. Additional attributes allow limiting the quantity of parts/assemblies which can be assigned at a certain position in the Variant BOM. Figure 1 shows the variant BOM of Agile PLM. Figure 1 Variant BOM in Agile PLM During the instantiation of the Variant BOM, the placeholders get replaced by specific variants of the parts and assemblies. The selection of the desired or appropriate variants is either done step by step by the user or by applying pre-defined configuration rules. As a result of the instantiation, an independent BOM will be created (Figure 2). Figure 2 Instantiated BOM in Agile PLM This kind of Variant BOM  can be used for „Assembled –To-Order“ type products as well as for „Engineered-to-Order“-type products. In case of “Assembled –To-Order” type products, typically the instantiation is done automatically with pre-defined configuration rules. For „Engineered- to-Order“-type products at least part of the product is selected manually to make use of customized parts/assemblies, that have been engineered according to the specific custom requirements. Template BOM The Template BOM is used for „Engineered-to-Order“-type products. It is another type of variant BOM. The engineer works in a flexible environment which allows him to build the most creative solutions. At the same time the engineer shall be guided to re-use existing solutions and it shall be assured that product variants of the same product family share the same base structure. The template BOM defines the basic structure of products belonging to the same product family. Let’s take a gearbox as an example. The customer specific configuration of the gearbox is influenced by several parameters (e.g. rpm range, transmitted torque), which are defined in the customer’s requirement document.  Figure 3 shows part of a Template BOM (yellow) and its relation to the product family hierarchy (blue).  Figure 3 Template BOM Every component of the Template BOM has links to the variants that have been engineeried so far for the component (depending on the level in the Template BOM, they are product variants, Assembly Variant or single part variants). This library of solutions, the so-called solution space, can be used by the engineers to build new product variants. In the best case, the engineer selects an existing solution variant, such as the gearbox shown in figure 3. When the existing variants do not fulfill the specific requirements, a new variant will be engineered. This new variant must be compliant with the given Template BOM. If we look at the gearbox in figure 3  it must consist of a transmission housing, a Connecting Plate, a set of Gears and a Planetary transmission – pre-assumed that all components are must have components. The new variant will enhance the solution space and is automatically available for re-use in future variants. The result of the instantiation of the Template BOM is a stand-alone BOM which represents the customer specific product variant. Modular BOM The concept of the modular BOM was invented in the automotive industry. Passenger cars are so-called „Assembled-to-Order“-products. The customer first selects the specific equipment of the car (so-called specifications) – for instance engine, audio equipment, rims, color. Based on this information the required parts will be determined and the customer specific car will be assembled. Certain combinations of specification are not available for the customer, because they are not feasible from technical perspective (e.g. a convertible with sun roof) or because the combination will not be offered for marketing reasons (e.g. steel rims with a sports line car). The modular BOM (yellow structure in figure 4) is defined in the context of a specific product family (in the sample it is product family „Speedstar“). It is the same modular BOM for the different types of cars of the product family (e.g. sedan, station wagon). The assembly or single parts of the car (blue nodes in figure 4) are assigned at the leaf level of the modular BOM. The assignment of assembly and parts to the modular BOM is enriched with a configuration rule (purple elements in figure 4). The configuration rule defines the conditions to use a specific assembly or single part. The configuration rule is valid in the context of a type of car (green elements in figure 4). Color specific parts are assigned to the color independent parts via additional configuration rules (grey elements in figure 4). The configuration rules use Boolean operators to connect the specifications. Additional consistency rules (constraints) may be used to define invalid combinations of specification (so-called exclusions). Furthermore consistency rules may be used to add specifications to the set of specifications. For instance it is important that a car with diesel engine always is build using the high capacity battery.  Figure 4 Modular BOM The calculation of the car configuration consists of several steps. First the consistency rules (constraints) are applied. Resulting from that specification might be added automatically. The second step will determine the assemblies and single parts for the complete structure of the modular BOM, by evaluating the configuration rules in the context of the current type of car. The evaluation of the rules for one component in the modular BOM might result in several rules being fulfilled. In this case the most specific rule (typically the longest rule) will win. Thanks to this approach, it is possible to add a specific variant to the modular BOM without the need to change any other configuration rules.  As a result the whole set of configuration rules is easy to maintain. Finally the color specific assemblies respective parts will be determined and the configuration is completed. Figure 5 Calculated Car Configuration The result of the car configuration is shown in figure 5. It shows the list of assemblies respective single parts (blue components in figure 5), which are required to build the customer specific car. Summary There are different approaches to variant management. Three different approaches have been presented in this article. At the end of the day, it is the type of the product which decides about the best approach.  For „Assembled to Order“-type products it is very likely that you can define the configuration rules and calculate the product variant automatically. Products of type „Engineered-to-Order“ ,however, need to be engineered. Nevertheless in the majority of cases, part of the product structure can be generated automatically in a similar way to „Assembled to Order“-tape products.  That said it is important first to analyze the product portfolio, in order to define the best approach to variant management.

    Read the article

  • cURL gets response with utf-8 BOM

    - by Reshat Belyalov
    In my script I send data with cURL, and enabled CURLOPT_RETURNTRANSFER. The response is json encoded data. When I'm trying to json_decode, it returns null. Then I found that response contains utf-8 BOM symbols at the beginning of string (). There is some experiments: $data = $data = curl_exec($ch); echo $data; the result is {"field_1":"text_1","field_2":"text_2","field_3":"text_3"} $data = $data = curl_exec($ch); echo mb_detect_encoding($data); result - UTF-8 $data = $data = curl_exec($ch); echo mb_convert_encoding($data, 'UTF-8', mb_detect_encoding($data)); // identical to echo mb_convert_encoding($data, 'UTF-8', 'UTF-8'); result - {"field_1":"text_1","field_2":"text_2","field_3":"text_3"} The one thing that helps is removing first 3 symbols: if (substr($data, 0, 3) == pack('CCC', 239, 187, 191)) { $data = substr($data, 3); } But what if there will be another BOM? So the question is: How to detect right encoding of cURL response? OR how to detect what BOM has arrrived? Or maybe how to convert the response with BOM? Thanks.

    Read the article

  • PHP Streaming CSV always adds UTF-8 BOM

    - by Mustafa Ashurex
    The following code gets a 'report line' as an array and uses fputcsv to tranform it into CSV. Everything is working great except for the fact that regardless of the charset I use, it is putting a UTF-8 bom at the beginning of the file. This is exceptionally annoying because A) I am specifying iso and B) We have lots of users using tools that show the UTF-8 bom as characters of garbage. I have even tried writing the results to a string, stripping the UTF-8 BOM and then echo'ing it out and still get it. Is it possible that the issue resides with Apache? If I change the fopen to a local file it writes it just fine without the UTF-8 BOM. header("Content-type: text/csv; charset=iso-8859-1"); header("Cache-Control: no-store, no-cache"); header("Content-Disposition: attachment; filename=\"report.csv\""); $outstream = fopen("php://output",'w'); for($i = 0; $i < $report-rowCount; $i++) { fputcsv($outstream, $report-getTaxMatrixLineValues($i), ',', '"'); } fclose($outstream); exit;

    Read the article

  • Python BOM error in Ascii file

    - by Intosia
    I have a wierd annoying problem with Python 2.6 I trying to run this file (and the other), on my Embedded Linux ARM board. http://svn.tuxisalive.com/software_suite_v3/smart-core/smart-server/trunk/TDSService.py I get this error File "tuxhttpserver.py", line 1 SyntaxError: encoding problem: with BOM I know that error is about the BOM bytes etc etc. BUT, there are NO BOM bytes, its plain Ascii. I checked with a Hexeditor, and the linux File command says its Ascii. Im freaking out here... The code worked fine on my Sheevaplug (also a ARM based system).

    Read the article

  • UTF-8 BOM Problem

    - by jsonx
    Hi, I am using Komodo Edit as an IDE. I have to encode some files as UTF-8 without BOM in Komodo. In my localhost and site there is no problem but on some sites i am seeing BOM sign and this is a terrible problem for AJAX-JSON response. Any advices? Thanks.

    Read the article

  • BOM in a PHP page auto generated by Wordpress

    - by Paolo63
    I admin two different blogs. They are both wordpress 2.8.6 (so they have exactly the same source code, plugins apart) but they are located on two different hosting platform (hostmonster.com and aruba.it). To explain my problem I've dumped with SmartSniff a session with each one of the sites. Here is the dump from hostmonster: GET /blog/paolo/ HTTP/1.1 Host: www.e-venturi.com Accept-Encoding: identity Accept-Language: en-us Accept: text/html, text/plain, text/xml, image/gif, image/x-xbitmap, image/x-icon,image/jpeg, image/pjpeg, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, */* User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;) HTTP/1.1 200 OK Date: Sat, 28 Nov 2009 23:47:38 GMT Server: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8l DAV/2 mod_auth_passthrough/2.1 FrontPage/5.0.2.2635 X-Powered-By: PHP/5.2.11 X-Pingback: http://www.e-venturi.com/blog/paolo/xmlrpc.php Vary: Accept-Encoding Transfer-Encoding: chunked Content-Type: text/html; charset=UTF-8 a6 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> and now from aruba: GET /blog/ HTTP/1.1 Host: www.cubanite.net Accept-Encoding: identity Accept-Language: en-us Accept: text/html, text/plain, text/xml, image/gif, image/x-xbitmap, image/x-icon,image/jpeg, image/pjpeg, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword, */* User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0;) HTTP/1.1 200 OK Date: Sat, 28 Nov 2009 23:49:19 GMT Server: Apache/2.2 X-Pingback: http://www.cubanite.net/blog/xmlrpc.php Vary: Accept-Encoding Transfer-Encoding: chunked Content-Type: text/html; charset=UTF-8 100b ...<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> (note: a6 and 100b are the packet size reported by SmartSniff) Ok, the big difference are the three dots in front of the <!DOCTYPE in aruba. They are the UTF-8 BOM (0xef 0xbb 0xbf). Being the same PHP source on both the servers, why does it appears only on one server ? The content is generated so the post author can't deliberately insert a BOM and I've verified the template to be BOM free too. Naturally there are different PHP and Apache versions on the servers... what can I check or set to diagnose and resolve the problem ? By the way I don't want the BOM. Many thanks in advance.

    Read the article

  • Create Text File Without BOM

    - by balexandre
    Hi guys, I tried this aproach without any success the code I'm using: // File name String filename = String.Format("{0:ddMMyyHHmm}", dtFileCreated); String filePath = Path.Combine(Server.MapPath("App_Data"), filename + ".txt"); // Process ProcessPBS pbs = new ProcessPBS(); pbs.CreatePBSInfoFile(pbslist, Convert.ToInt64(filename)); // Save file Encoding utf8WithoutBom = new UTF8Encoding(true); TextWriter tw = new StreamWriter(filePath, false, utf8WithoutBom); foreach (string s in pbs.GeneratedFile.ToArray()) tw.WriteLine(s); tw.Close(); // Push Generated File into Client Response.Clear(); Response.ContentType = "application/vnd.text"; Response.AppendHeader("Content-Disposition", "attachment; filename=" + filename + ".txt"); Response.TransmitFile(filePath); Response.End(); the result: It's writing the BOM no matter what, and special chars (like Æ Ø Å) are not correct :-/ I'm stuck! I can't create a file using UTF-8 as Encoding and 8859-1 as CharSet :-( Anyone can help me find the light in the tunnel? All help is greatly appreciated, thank you!

    Read the article

  • UTF-8 BOM signature in PHP files

    - by skidding
    I was writing some commented PHP classes and I stumbled upon a problem. My name (for the @author tag) ends up with a ? (which is a UTF-8 character, ...and a strange name, I know). Even though I save the file as UTF-8, some friends reported that they see that character totally messed up (È™). This problem goes away by adding the BOM signature. But that thing troubles me a bit, since I don't know that much about it, except from what I saw on Wikipedia and on some other similar questions here on SO. I know that it adds some things at the beginning of the file, and from what I understood it's not that bad, but I'm concerned because the only problematic scenarios I read about involved PHP files. And since I'm writing PHP classes to share them, being 100% compatible is more important than having my name in the comments. But I'm trying to understand the implications, should I use it without worrying? or are there cases when it might cause damage? When? Thanks!

    Read the article

  • UTF-8 BOM in php response to mootools xmlhttprequest

    - by Jimmy
    Hi, I'm writing my first little AJAX-enabled Joomla component. I'm using mootools. I got a xmlhttprequest to contact my Joomla component, and the component returns a response - just plain text echoed by php, like echo 'Hello World!'; It's all working fine, except wireshark tells me that the response is prepended with \357\273\277\357\273\277 when it gets read by the javascript on the client side. This shows up as a little square before the response in an alert box that the script shows. I don't explicitly set the encoding on the xmlhttprequest; mootools docs say that it defaults to UTF8. What's the right way to handle this? Should I be setting the encoding on the request? Mime type? Should the javascript get rid of it? I'm not planning to have any characters requiring UTF8 in the response, so using plain old ascii would be ok for me too. Thanks

    Read the article

  • Typical text encoding+BOM, and EOL behavior on mobile devices

    - by Dan W
    Typical things to worry about when dealing with text are the BOM/signature, encoding, and the end of line (EOL) char/chars. I know that Windows often favours \r\n (CR+LF) and Mac/Linux favours \n (LF), but how about mobile devices such as the iPhone and Android? Do typical apps on those platforms favour one or the other? Also, which text encodings are mobiles most likely to use - UTF-8, iso-8859-1, or even Windows 1252 (or other default codepage) or maybe even UTF-16? And if they use UTF-8/16, are they likely to need (or require not having) a BOM/signature? What is the typical behavior here?

    Read the article

  • How do you get Matlab to write the BOM (byte order markers) for UTF-16 text files?

    - by Richard Povinelli
    I am creating UTF16 text files with Matlab, which I am later reading in using Java. In Matlab, I open a file called fileName and write to it as follows: fid = fopen(fileName, 'w','n','UTF16-LE'); fprintf(fid,"Some stuff."); In Java, I can read the text file using the following code: FileInputStream fileInputStream = new FileInputStream(fileName); Scanner scanner = new Scanner(fileInputStream, "UTF-16LE"); String s = scanner.nextLine(); Here is the hex output: Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 00000000 73 00 6F 00 6D 00 65 00 20 00 73 00 74 00 75 00 66 00 66 00 s.o.m.e. .s.t.u.f.f. The above approach works fine. But, I want to be able to write out the file using UTF16 with a BOM to give me more flexibility so that I don't have to worry about big or little endian. In Matlab, I've coded: fid = fopen(fileName, 'w','n','UTF16'); fprintf(fid,"Some stuff."); In Java, I change the code to: FileInputStream fileInputStream = new FileInputStream(fileName); Scanner scanner = new Scanner(fileInputStream, "UTF-16"); String s = scanner.nextLine(); In this case, the string s is garbled, because Matlab is not writing the BOM. I can get the Java code to work just fine if I add the BOM manually. With the added BOM, the following file works fine. Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 00000000 FF FE 73 00 6F 00 6D 00 65 00 20 00 73 00 74 00 75 00 66 00 66 00 ÿþs.o.m.e. .s.t.u.f.f. How can I get Matlab to write out the BOM? I know I could write the BOM out separately, but I'd rather have Matlab do it automatically. Addendum I selected the answer below from Amro because it exactly solves the question I posed. One key discovery for me was the difference between the Unicode Standard and a UTF (Unicode transformation format) (see http://unicode.org/faq/utf_bom.html). The Unicode Standard provides unique identifiers (code points) for characters. UTFs provide mappings of every code point "to a unique byte sequence." Since all but a handful of the characters I am using are in the first 128 code points, I'm going to switch to using UTF-8 as Romeo suggests. UTF-8 is supported by Matlab (The warning shown below won't need to be suppressed.) and Java, and for my application will generate smaller text files. I suppress the Matlab warning Warning: The encoding 'UTF-16LE' is not supported. with warning off MATLAB:iofun:UnsupportedEncoding;

    Read the article

  • java: can I convert strings with String.getBytes() without the BOM?

    - by Cheeso
    Suppose I have this code: String encoding = "UTF-16"; String text = "[Hello StackOverflow]"; byte[] message= text.getBytes(encoding); If I display the byte array in message, the result is: 0000 FE FF 00 5B 00 48 00 65 00 6C 00 6C 00 6F 00 20 ...[.H.e.l.l.o. 0010 00 53 00 74 00 61 00 63 00 6B 00 4F 00 76 00 65 .S.t.a.c.k.O.v.e 0020 00 72 00 66 00 6C 00 6F 00 77 00 5D .r.f.l.o.w.] As you can see, there's a BOM in the beginning. How can I: generate a UTF-16 byte array that lacks a BOM ? convert from a byte array that contains UTF=16 chars but lacks a BOM, back to a string?

    Read the article

  • java: can I convert strings to byte arrays, without a BOM?

    - by Cheeso
    Suppose I have this code: String encoding = "UTF-16"; String text = "[Hello StackOverflow]"; byte[] message= text.getBytes(encoding); If I display the byte array in message, the result is: 0000 FE FF 00 5B 00 48 00 65 00 6C 00 6C 00 6F 00 20 ...[.H.e.l.l.o. 0010 00 53 00 74 00 61 00 63 00 6B 00 4F 00 76 00 65 .S.t.a.c.k.O.v.e 0020 00 72 00 66 00 6C 00 6F 00 77 00 5D .r.f.l.o.w.] As you can see, there's a BOM in the beginning. How can I: generate a UTF-16 byte array that lacks a BOM, from a string? convert from a byte array that contains UTF-16 chars but lacks a BOM, back to a string?

    Read the article

  • Adeus ano bom – bem vindo, melhor ainda.

    - by anobre
    Olá pessoal! Este ano de 2010 foi realmente muito bom. A NBR cresceu consideravelmente, eu tive algumas mudanças no meu trabalho, me casei, comprei um apartamento e tudo mais. Definitivamente, este ano ficará marcado para sempre. Fiz diversos amigos, dentro da comunidade Microsoft também. O melhor ainda é que as perspectivas para 2011 são ainda mais positivas. E pelo que estou vendo, este sentimento é comum a muitos amigos por aí, o que me deixa mais feliz ainda. Sem mais, desejo a todos um feliz ano de 2011, que muitos objetivos e conquistas sejam alcançados. Indepentende de tecnologias, linguagens ou religiões, que cada um faça o seu melhor, para garantir o melhor a sua volta. Espero que todos sigam duas palavras: MUDANÇAS e EMPREENDEDORISMO. Um forte abraço a todos e até ano que vem!

    Read the article

  • PHP: json_decode dumping NULL, BOM not found

    - by SerEnder
    I've been trying to find out why this 'json_encode'd string isn't parsing out correctly, and came across previously answered questions that had the UTF BOM sequence that was throwing the error, but didn't help me here. Here's the code that isn't currently working: //Decode the notes attached to the sig $aNotes = json_decode($rule->getNotes(),true); $bom = pack("CCC",0xef,0xbb,0xbf); if(0 == strncmp($rule->getNotes(),$bom,3)) { print('BOM detected - json encoding in UTF-8<br/>'); } else { print('BOM NOT detected - json encoding correctly<br/>'); } print('rule->getNotes:<br/>' . $rule->getNotes() .'<br/>'); var_dump($aNotes); Which generates this result: BOM NOT detected - json encoding correctly rule->getNotes: [{"lDate":"Unknown","sAuthor":"Unknown","sNote":"This is a general purpose Russian spam rule that matches anything starting with 2, 3 or 4 hex digits followed by a domain name ending with .ru -RSK 2010-05-10"},{"lDate":"1295031463082","sAuthor":"Drew Thorstenson","sNote":"this is Ryan's ru rule"}] NULL I've run it through JSON Lint, which said it was valid, and An Online JSON Parser which parsed it correctly too. Any insight would be greatly appreciated.

    Read the article

  • Handling UTF-8 with BOM in HTTP

    - by Alois Mahdal
    Say I have a script which at some point serves a plain text file as a content (right after "\n\n"). These files are provided by users, but I can expect they will be UTF-8. So I hard-wire Content-Type: text/plain; charset=UTF-8. But while I can teach users to save everything in UTF-8, I can't be very sure that the files will be without BOM ("\xEE\xBB\xBF"), as at least on Windows, this is not very clearly distinguished in common plain text editors and not every one of them uses the same default. So what about these files created on Windows, where they may/may not start with BOM? Should/will server or UA get rid of this debris for me? Or is it my task to prepare clean UTF-8, i.e. open each file and check whether BOM needs to be removed?

    Read the article

  • Saving a file in a CSV type in Excel always removes the BOM

    - by rickp
    I've been trying to find a reasonable solution/explanation (unsuccessfully) to find out why Excel defaults to removing the BOM when saving a file to the CSV type. Please forgive me if you find this a duplicate of this question. This handles reading CSV files with non-ASCII encoding, but it doesn't cover saving the file back out (which is where the biggest issue lies). Here is my current situation (which I'm going to gather is common among localized software dealing with Unicode characters and a CSV format): We export data to a CSV format using UTF-16LE, ensuring the BOM is set (0xFFFE). We validate after the file is generated with a Hex editor to ensure it was set correctly. Open the file in Excel (for this example we're exporting Japanese characters) and witness that Excel handles loading the file with the correct encoding. Attempts to save this file will prompt you with a warning message indicating that the file may contain features that may not be compatible with Unicode encoding, but asks if you'd like to save anyway. If you select the Save As dialog, it will immediately ask you to save the file as "Unicode Text" rather than CSV. If you select the "CSV" extension and save the file it removes the BOM (obviously along with all the Japanese characters). Why would this happen? Is there a solution to this problem, or is this a known 'bug'/limitation of Excel? Additionally (as a side issue) it appears that Excel, when loading UTF-16LE encoded CSV files, only uses TAB delimiters. Again, is this another known 'bug'/limitation of Excel?

    Read the article

  • Saving a file in a CSV type in Excel always removes the BOM

    - by rickp
    I've been trying to find a reasonable solution/explanation (unsuccessfully) to find out why Excel defaults to removing the BOM when saving a file to the CSV type. Please forgive me if you find this a duplicate of this question. This handles reading CSV files with non-ASCII encoding, but it doesn't cover saving the file back out (which is where the biggest issue lies). Here is my current situation (which I'm going to gather is common among localized software dealing with Unicode characters and a CSV format): We export data to a CSV format using UTF-16LE, ensuring the BOM is set (0xFFFE). We validate after the file is generated with a Hex editor to ensure it was set correctly. Open the file in Excel (for this example we're exporting Japanese characters) and witness that Excel handles loading the file with the correct encoding. Attempts to save this file will prompt you with a warning message indicating that the file may contain features that may not be compatible with Unicode encoding, but asks if you'd like to save anyway. If you select the Save As dialog, it will immediately ask you to save the file as "Unicode Text" rather than CSV. If you select the "CSV" extension and save the file it removes the BOM (obviously along with all the Japanese characters). Why would this happen? Is there a solution to this problem, or is this a known 'bug'/limitation of Excel? Additionally (as a side issue) it appears that Excel, when loading UTF-16LE encoded CSV files, only uses TAB delimiters. Again, is this another known 'bug'/limitation of Excel?

    Read the article

  • Working with utf-8 files in Eclipse.

    - by Pablo Cabrera
    Quite straight forward question. Is there a way to configure Eclipse to work with text files encoded with utf-8 with and without the BOM? So far I've used eclipse with utf-8 encoding and it works, but when I try to edit a file generated by another editor that includes the BOM, Eclipse doesn't handle it properly, it 'shows an invisible character' at the begining of the file (the BOM). Is there a way to make Eclipse understand utf-8 encoded files with BOM?

    Read the article

  • JAX-WS errors when SOAP body contains UTF-8 BOM

    - by Vinny Carpenter
    I have developed a Web Service using JAX-WS (v2.1.3 - Sun JDK 1.6.0_05) deployed on WebLogic 10.3 that works just fine when I use a Java client or SoapUI or other Web Services testing tools. I need to consume this service using 2005 Microsoft SQL Server Reporting Services and I get the following error Couldn't create SOAP message due to exception: XML reader error: unexpected character content SEVERE: Couldn't create SOAP message due to exception: XML reader error: unexpected character content: "?" com.sun.xml.ws.protocol.soap.MessageCreationException: Couldn't create SOAP message due to exception: XML reader error: unexpected character content: "?" at com.sun.xml.ws.encoding.SOAPBindingCodec.decode(SOAPBindingCodec.java:292) at com.sun.xml.ws.transport.http.HttpAdapter.decodePacket(HttpAdapter.java:276) at com.sun.xml.ws.transport.http.HttpAdapter.access$500(HttpAdapter.java:93) at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:432) at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:244) at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:134) at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.doGet(WSServletDelegate.java:129) at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.doPost(WSServletDelegate.java:160) at com.sun.xml.ws.transport.http.servlet.WSServlet.doPost(WSServlet.java:75) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227) at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125) at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292) at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175) at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3498) at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) at weblogic.security.service.SecurityManager.runAs(Unknown Source) at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180) at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086) at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406) at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201) at weblogic.work.ExecuteThread.run(ExecuteThread.java:173) Caused by: com.sun.xml.ws.streaming.XMLStreamReaderException: XML reader error: unexpected character content: "?" at com.sun.xml.ws.streaming.XMLStreamReaderUtil.nextElementContent(XMLStreamReaderUtil.java:102) at com.sun.xml.ws.encoding.StreamSOAPCodec.decode(StreamSOAPCodec.java:174) at com.sun.xml.ws.encoding.StreamSOAPCodec.decode(StreamSOAPCodec.java:296) at com.sun.xml.ws.encoding.StreamSOAPCodec.decode(StreamSOAPCodec.java:128) at com.sun.xml.ws.encoding.SOAPBindingCodec.decode(SOAPBindingCodec.java:287) ... 22 more If I use a HTTP proxy to sniff out what SSRS is sending to JAX-WS, I see EF BB BF as the beginning of the post body and JAX-WS doesn't like that. If I remove the special characters and resubmit the request using Fiddler, then the web-service invocation works. Why does JAX-WS blow up with the standard UTF-8 BOM? Is there a workaround to get past this issue? Any suggestions would be greatly appreciated. Thanks --Vinny

    Read the article

  • is unicode( codecs.BOM_UTF8, "utf8" ) necessary in Python 2.7/3?

    - by Brian M. Hunt
    In a code review I came across the following code that contains the following: # Python bug that renders the unicode identifier (0xEF 0xBB 0xBF) # as a character. # If untreated, it can prevent the page from validating or rendering # properly. bom = unicode( codecs.BOM_UTF8, "utf8" ) r = r.replace(bom, '') This is in a function that passes a string to Response object (Django or Flask). Is this still a bug that needs this fix in Python 2.7 or 3? Something tells me it isn't, but I thought I'd ask because I don't know this problem very well. Thanks for reading.

    Read the article

1 2 3 4 5 6 7  | Next Page >