How to find malformed - corrupted - dos - BOMByte Files in Linux

Posted by Syquus on Server Fault See other posts from Server Fault or by Syquus
Published on 2010-09-09T19:20:18Z Indexed on 2011/01/15 12:54 UTC
Read the original article Hit count: 166

Filed under:

linux

|

vim

|

encoding

|

text

|

characterset

I've several problems maintaining large production servers, in which some developers drop files from Windows environments, sometime with BOM-bytes (We use UTF8, and no need for that), causing lots of troubles.

Other times, I got a "no end of line" and "[DOS]" labels when vim-editing files directly on the server.

I recently discovered how to find for the bom byte, and how to delete it in a batch script. What about illegal bytes, bad EOLs? Is it safe to use DOS Text Files on a linux environment? Any drawbacks If I use to convert them with dos2unix cmd ?

Regards

© Server Fault or respective owner

Related posts about linux

apt-get install and update fail

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I've got a problem with apt-get update and apt-get install ... commands . every time update or installing fails and errors are : Get:1 http://dl.google.com stable Release.gpg [198B] Ign http://dl.google.com/linux/chrome/deb/ stable/main Translation-en_US Get:2 http://dl… >>> More
kernel module compiling error

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
sh@ubuntu:/home/ccpp/helloworld$ make gcc-4.6 -O2 -DMODULE -D_KERNEL_ -W -Wall -Wstrict-prototypes -Wmissing-prototypes -isystem /lib/modules/`uname -r`/build/include -c -o hello-1.o hello-1.c hello-1.c:4:0: warning: "MODULE" redefined [enabled by default] <command-line>:0:0: note: this is… >>> More
Build-Essentials installation failing

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am having trouble accessing the several critical header files that show to be a part of the build process. The "Ubuntu Software Center" shows "Build Essentials" as installed: Next I did the following two commands, which did not improve the problem: ~$ sudo apt-get install build-essential [sudo]… >>> More
Updating Debian kernel

as seen on Super User - Search for 'Super User'
I'm trying to update my Debian machine to 2.6.32-46 (which is the new stable). However, after doing apt-get update my apt-cache search linux-image shows me: linux-headers-2.6.32-5-486 - Header files for Linux 2.6.32-5-486 linux-headers-2.6.32-5-686-bigmem - Header files for Linux 2.6.32-5-686-bigmem linux-headers-2… >>> More
Serial connection over a single USB cable (Windows to linux, or linux to linux)

as seen on Server Fault - Search for 'Server Fault'
I'm helping out with a project for an embedded device that only has USB and no serial. This device is running Linux. These days, when we need to connect to a serial port on a device we typically use a USB to serial adapter (on something like a phone system or a load balancing device, etc). I would… >>> More

Related posts about vim

Vim: Use different ~/.vim/plugin/ directories for different versions of vim?

as seen on Super User - Search for 'Super User'
Like many of you, my custom Vim configuration is stored in my ~/.vimrc, with the plugins, colors, etc. stored under ~/.vim/plugins, ~/.vim/colors, etc. I want to share a single Vim configuration among many servers. Some of these servers run Vim 7, some run the older Vim 6. Most Vim plugins are intended… >>> More
Why isn't the scripts in my autoload folder being executed in Vim?

as seen on Super User - Search for 'Super User'
I'm trying to use Pathogen to manage my Vim extensions. My bundle folder looks like this: .../bundle/ +-- vim-pathogen ¦ +-- autoload ¦ +-- pathogen.vim +-- vim-smoothscroll +-- autoload +-- smooth_scroll.vim And my vimrc file includes this: let s:root = fnamemodify(resolve(expand(":p"))… >>> More
Cygwin vim doesn't load files in runtimepath

as seen on Super User - Search for 'Super User'
I created a custom syntax file, and but none of the files in $VIMRUNTIME seem to load. I followed these pieces of the documentation: http://vimdoc.sourceforge.net/htmldoc/filetype.html#new-filetype http://vimdoc.sourceforge.net/htmldoc/syntax.html#mysyntaxfile When I do :echo &runtimepath I… >>> More
Debugging matchit plugin in vim (under Cygwin)

as seen on Stack Overflow - Search for 'Stack Overflow'
The "matchit" plugin for vim is supposed to allow you to use the % key to jump between matching start/end tags when editing HTML, as well as /* and */ comment delimiters when editing other kinds of code. I've followed the exact instructions in ":help matchit", but % still doesn't work for me. It… >>> More
Vim: How to execute commands stored in a textfile opened in Vim

as seen on Super User - Search for 'Super User'
I store, commands found at http://www.commandlinefu.com/commands/browse, in a text file. If I open this text file in Vim and I want to execute the command where the cursor is actually located I do copy the line with Y, switching to command mode with : and !CTRL+r* Is there a more "lazy" possibility ;) >>> More