Crlf: Difference between revisions
Jump to navigation
Jump to search
(Created page with 'Everything about ascii file line termination issue (LF, CR/LF, CR...) == Detection == * Hex dump <source lang=bash> hd <file> # 00000000 48 65 6c 6c 6f 0d 0a 48 65 6c 6c 6f 0d…') |
|||
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
Everything about ascii file line termination issue (LF, CR/LF, CR...) |
Everything about ascii file line termination issue (LF, CR/LF, CR...) |
||
== Reference == |
|||
* [http://vim.wikia.com/wiki/File_format File format in Vim] |
|||
== Miscellaneous == |
|||
* Normal Linux line ending is LF |
|||
* Normal DOS line ending is CR-LF |
|||
* In Bash, insert special character with {{kbctrl|V}}. For instance {{kbctrl|V}} {{kbctrl|M}} will insert a CR character |
|||
** Alternatively, use something like <code>r="$(printf "\r")"</code>, |
|||
** Or use <code>echo $'\r'</code> |
|||
== Detection == |
== Detection == |
||
| Line 11: | Line 21: | ||
* Use ''<tt>file</tt>'': |
* Use ''<tt>file</tt>'': |
||
<source lang= |
<source lang=bash> |
||
file crlf.txt cr_and_lf.txt lf.txt |
file crlf.txt cr_and_lf.txt lf.txt |
||
# crlf.txt: ASCII text, with CRLF line terminators |
# crlf.txt: ASCII text, with CRLF line terminators |
||
# cr_and_lf.txt: ASCII text, with CRLF, LF line terminators |
# cr_and_lf.txt: ASCII text, with CRLF, LF line terminators |
||
# lf.txt: ASCII text |
|||
find . -print0 | xargs -0 file | grep CR # Search all files |
|||
</source> |
|||
* Using GREP: |
|||
<source lang=bash> |
|||
grep -IUr --color $'\r' # Add -l to have list of files |
|||
find . -name "*.[ch]" -print0 | xargs -0 grep -IUl $'\r' # Prefilter files to look at |
|||
</source> |
|||
== Conversion == |
|||
Reference: [http://www.cyberciti.biz/faq/howto-unix-linux-convert-dos-newlines-cr-lf-unix-text-format/] |
|||
* Using package '''<tt>dos2unix</tt>''': |
|||
<source lang="bash"> |
|||
dos2unix <file> |
|||
unix2dos <file> |
|||
mac2unix <file> |
|||
unix2mac <file> |
|||
</source> |
|||
For instance, to fix all CR/CRLF issues, and remove also any trailing blanks (i.e. sanitizing all sources): |
|||
<source lang="bash"> |
|||
# Convert CRLF to LF |
|||
find . -name '*.[chCH]' -print0 | xargs -0 dos2unix |
|||
# ... twice in case we had "CRCRLF" in source files (from a bad LF->CRLF conversion) |
|||
find . -name '*.[chCH]' -print0 | xargs -0 dos2unix |
|||
# Convert CR to LF |
|||
find . -name '*.[chCH]' -print0 | xargs -0 mac2unix |
|||
# Remove trailing blanks |
|||
find . -name '*.[chCH]' -print0 | xargs -0 -n 1 sed -i -r 's/ +$//' |
|||
</source> |
|||
* Using command '''<tt>tr</tt>''': |
|||
<source lang="bash"> |
|||
tr -d '\r' < file # Delete RETURN (i.e. CR, ^M, ascii 13) |
|||
tr -d '\n' < file # Delete NEWLINE (i.e. LF, ^J, ascii 10) |
|||
</source> |
|||
* Using '''Vim''': |
|||
<source lang="bash"> |
|||
vim file |
|||
</source> |
|||
<source lang="vim"> |
|||
:set ff=unix |
|||
:wq |
|||
</source> |
|||
* Using '''Perl''': |
|||
<source lang="bash"> |
|||
perl -pi -e 's/\r\n/\n/g' input.file # CRLF to LF |
|||
</source> |
|||
* Using '''Sed''': |
|||
<source lang="bash"> |
|||
sed 's/$/^M/" input.txt > output.txt # LF to CRLF |
|||
sed 's/^M$//' input.txt > output.txt # CRLF to LF |
|||
</source> |
</source> |
||
Latest revision as of 09:59, 14 October 2011
Everything about ascii file line termination issue (LF, CR/LF, CR...)
Reference
Miscellaneous
- Normal Linux line ending is LF
- Normal DOS line ending is CR-LF
- In Bash, insert special character with ^V. For instance ^V ^M will insert a CR character
- Alternatively, use something like
r="$(printf "\r")", - Or use
echo $'\r'
- Alternatively, use something like
Detection
- Hex dump
hd <file>
# 00000000 48 65 6c 6c 6f 0d 0a 48 65 6c 6c 6f 0d 0a 48 65 |Hello..Hello..He|
# 00000010 6c 6c 6f 0d 0a 48 65 6c 6c 6f 0d 0a |llo..Hello..|
# 0000001c
- Use file:
file crlf.txt cr_and_lf.txt lf.txt
# crlf.txt: ASCII text, with CRLF line terminators
# cr_and_lf.txt: ASCII text, with CRLF, LF line terminators
# lf.txt: ASCII text
find . -print0 | xargs -0 file | grep CR # Search all files
- Using GREP:
grep -IUr --color $'\r' # Add -l to have list of files
find . -name "*.[ch]" -print0 | xargs -0 grep -IUl $'\r' # Prefilter files to look at
Conversion
Reference: [1]
- Using package dos2unix:
dos2unix <file>
unix2dos <file>
mac2unix <file>
unix2mac <file>
For instance, to fix all CR/CRLF issues, and remove also any trailing blanks (i.e. sanitizing all sources):
# Convert CRLF to LF
find . -name '*.[chCH]' -print0 | xargs -0 dos2unix
# ... twice in case we had "CRCRLF" in source files (from a bad LF->CRLF conversion)
find . -name '*.[chCH]' -print0 | xargs -0 dos2unix
# Convert CR to LF
find . -name '*.[chCH]' -print0 | xargs -0 mac2unix
# Remove trailing blanks
find . -name '*.[chCH]' -print0 | xargs -0 -n 1 sed -i -r 's/ +$//'
- Using command tr:
tr -d '\r' < file # Delete RETURN (i.e. CR, ^M, ascii 13)
tr -d '\n' < file # Delete NEWLINE (i.e. LF, ^J, ascii 10)
- Using Vim:
vim file
:set ff=unix
:wq
- Using Perl:
perl -pi -e 's/\r\n/\n/g' input.file # CRLF to LF
- Using Sed:
sed 's/$/^M/" input.txt > output.txt # LF to CRLF
sed 's/^M$//' input.txt > output.txt # CRLF to LF