Crlf

Everything about ascii file line termination issue (LF, CR/LF, CR...)

Reference

• File format in Vim

Miscellaneous

• Normal Linux line ending is LF
• Normal DOS line ending is CR-LF
• In Bash, insert special character with ^V. For instance ^V ^M will insert a CR character
  • Alternatively, use something like r="$(printf "\r")",
  • Or use echo $'\r'

Detection

• Hex dump

hd <file>
# 00000000  48 65 6c 6c 6f 0d 0a 48  65 6c 6c 6f 0d 0a 48 65  |Hello..Hello..He|
# 00000010  6c 6c 6f 0d 0a 48 65 6c  6c 6f 0d 0a              |llo..Hello..|
# 0000001c

• Use file:

file crlf.txt cr_and_lf.txt lf.txt
# crlf.txt:       ASCII text, with CRLF line terminators
# cr_and_lf.txt:  ASCII text, with CRLF, LF line terminators
# lf.txt:         ASCII text
find . -print0 | xargs -0 file | grep CR                  # Search all files

• Using GREP:

grep -IUr --color $'\r'                                   # Add -l to have list of files
find . -name "*.[ch]" -print0 | xargs -0 grep -IUl $'\r'  # Prefilter files to look at

Conversion

Reference: [1]

• Using package dos2unix:

dos2unix <file>
unix2dos <file>
mac2unix <file>
unix2mac <file>

For instance, to fix all CR/CRLF issues, and remove also any trailing blanks (i.e. sanitizing all sources):

# Convert CRLF to LF
find . -name '*.[chCH]' -print0 | xargs -0 dos2unix
# ... twice in case we had "CRCRLF" in source files (from a bad LF->CRLF conversion)
find . -name '*.[chCH]' -print0 | xargs -0 dos2unix
# Convert CR to LF
find . -name '*.[chCH]' -print0 | xargs -0 mac2unix
# Remove trailing blanks
find . -name '*.[chCH]' -print0 | xargs -0 -n 1 sed -i -r 's/ +$//'

• Using command tr:

tr -d '\r' < file            # Delete RETURN (i.e. CR, ^M, ascii 13)
tr -d '\n' < file            # Delete NEWLINE (i.e. LF, ^J, ascii 10)

• Using Vim:

vim file

:set ff=unix
:wq

• Using Perl:

perl -pi -e 's/\r\n/\n/g' input.file              # CRLF to LF

• Using Sed:

sed 's/$/^M/" input.txt > output.txt              # LF to CRLF
sed 's/^M$//' input.txt > output.txt              # CRLF to LF