Silent failure of shell scripts (.sh) on Linux and in Docker

Tue, 14 Feb 2023 by garethbrown

Windows / Unix / Linux style line endings

A common reason for the apparent failure of shell scripts used by Docker containers launched from within WSL is Windows style line endings (CRLF) where the source files are accessed from the mounted Windows file system. This has has been a common issue for me since using WSL2 and Docker more in a Windows development workflow.

The issue is that for a default Git client configurations, files are checked out on Windows PCs with Windows style line endings (CRLF), with the line endings being converted back to Linux/Unix style line endings when checking back in. Explanation of this issue is well covered on this stackoverflow post. This post addresses mitigations for working in environments and around WSL2 where this is a common issue.

When this occurs in WSL based development workflows

If you check out a Git respository on Windows, and work with files via a WSL2 hosted Linux distribution, then you may find that you are working with Windows style line endings (CRLF) in a system (Linux) that expects Linux style line endings (LF). There are a few options for dealing with this:

Clone and manage the repository directly from a shell into the WSL Linux instance. Here code will be checked out with LF line endings, and be checked back in with LF line endings by default.
Run a utility from the WSL Linux instance called dos2unix to change the line endings on files where necessary.
Configure Git globally to change line ending modification behaviour.
Use a .gitattributes file to ensure that certain files always use LF (Linux style) line endings.

Mitigating with .gitattributes

I've found that the .gitattributes option works best for me. I've found that checking repositories out in two places (WSL Linux and Windows) to be an awkward development workflow. Working exclusively within WSL also feels awkward. I'd be happy to move over completely to Linux, except too many of my clients provide Windows development machines for project work. I also often run docker containers that depend on shell scripts mapped into the container volume.

Since the difference in line endings only seems to be problematic for shell scripts (.sh), I've opted to add a .gitattributes file with configuration for .sh scripts so that line endings are always checked in and out Linux style (LF), regardless of the global Git client configuration.

*.sh text eol=lf

This means I can start Docker containers using files from the mounted Windows directory (/mnt/c/...) within a WSL Linux shell, without worrying that the script may be corrupted with CRLF line endings.