What I'm Trying To Do
I'm trying to use FAKE's TemplateHelper to instantiate some template files which become shell scripts (both Unix and DOS). I was doing that in a FAKE task like this:
CopyFile (buildApp @@ product + ".sh") (srcTem @@ "mono-launcher.sh")
processTemplates substs [(buildApp @@ product + ".sh")]
where substs was the var->value mappings to substitute, and srcTem and buildApp are the template source directories and the build output directories, respectively.
What Went Wrong
I encountered 3 issues:
First, TemplateHelper.processTemplates was writing a UTF-8 BOM at the front of the generated script. On Unix at least, this messes up the shebang line and gives an error message when the script is run. An octal dump of the 1st line of the original template and the generated file show the appearance of these 3 troublesome bytes:
$ head -1 ./src/templates/mono-launcher.sh | od -c
0000000 # ! / b i n / s h \n
$ head -1 ./build/app/Hello-Fsharp.sh | od -c
0000000 357 273 277 # ! / b i n / s h \n
Second, there was no way to say the Unix sh scripts generated should have Unix EOLs in them, whereas the Windows .bat scripts should have DOS EOLs.
Third, there was no apparent way to say that the resulting files should have world read and execute permissions.
This question is about whether anyone knows how to deal with those issues, at least in a way better than what I came up with below.
I did a bit of spelunking in the TemplateHelper sources and the environs. I discovered processTemplate calls WriteFile which calls WriteToFile which uses the value encoding defined in EnvironmentHelper as System.Text.Encoding.Default, which on my system is UTF-8. That's well enough, but there's no apparent way to pass in a custom encoding, e.g., ASCIIEncoding.
What I Did About It
So... absent any way to persuade TemplateHelper to do what I want, I replaced it with the following:
/// <summary>Copy a template file from source to destination, making string substitutions
/// along the way and allowing for different file encodings at each end.</summary>
///
/// <remarks>This only exists because TemplateHelper put UTF-8 BOM at the front, breaking
/// script shebang lines. The EOL choice here works from Unix but needs testing on
/// Windows (will it double up on returns if the build machine is Window?), and the
/// read/execute permission setting is iffy. </remarks>
///
/// <param name="substs">IDictionary of key, value string replacements to make</param>
/// <param name="fromEncoding">Encoding of fromTemplate</param>
/// <param name="fromTemplate">file with the source template</param>
/// <param name="toEncoding">Encoding of toFile</param>
/// <param name="toFile">file to which the instantiated template gets written</param>
/// <param name="readExecute">whether to mark toFile as readable & executable by all</param>
///
/// <returns>Nothing of interest.</returns>
let processTemplate substs fromEncoding fromTemplate toEncoding toFile unixEOL readExecute =
let ReadTemplate encoding fileName = // Read lines of template file, but lazily:
File.ReadLines(fileName, encoding) // start processing before entirety is read
let SubstituteVariables substs = // Substitute variables in seq of line strs
Seq.map (fun (line : string) -> // Loop over lines
substs // Fold in effect of each key, val pair
|> Seq.fold (fun (sb : StringBuilder) (KeyValue(k : string, v : string)) ->
sb.Replace(k, v)) // Generates new StringBuilder w/each var
(new StringBuilder(line)) // Initial accumulator is original line
|> toText) // Convert final StringBuilder back to text
let MaybeAddReturn = // Add extra return if not Unix EOLs
Seq.map (fun line -> if unixEOL then line else line + "\r") // *** Test if works on Windows?
let WriteTemplate encoding fileName (lines : seq<string>) =
File.WriteAllLines(fileName, lines, encoding) // Write out altered lines (eol chosen how?)
fileName // Return filename for next step in pipeline
let MaybeSetReadExecute fileName = // Platform neutral, mark read + exec perms
if readExecute then // Only do this if asked
if isMono then // We're running on Mono
Shell.Exec("chmod", "a+rx " + fileName, here) |> ignore // *** Horrible kludge?
else // Not under Mono: do it the .Net way
// *** Equiv kludge: Shell.Exec("icacls", fileName + " /grant S-1-1-0:RX", here) |> ignore
let fs = File.GetAccessControl(fileName) // Mac OSX: System.NotSupportedException
fs.AddAccessRule(new FileSystemAccessRule( //
new SecurityIdentifier(WellKnownSidType.WorldSid, null),
FileSystemRights.ReadAndExecute,
AccessControlType.Allow)) // *** Seems pretty complex for such a simple
File.SetAccessControl(fileName, fs) // task as just setting rx permissions!
let shortClassName obj = // Just class name, no namespacey qualifiers
let str = obj.ToString() in str.Substring(str.LastIndexOf('.') + 1)
logfn " Template %s (%s) generating %s (%s, %s EOLs%s)."
fromTemplate (shortClassName fromEncoding) // Where we're coming from,
toFile (shortClassName toEncoding) // where we're going,
(if unixEOL then "Unix" else "DOS") // and in what condition we propose to
(if readExecute then ", read/execute" else "") // arrive there
fromTemplate // Start with the input template filename
|> ReadTemplate fromEncoding // read it in lazily (enumeration of lines)
|> SubstituteVariables substs // substitute variables in each line
|> MaybeAddReturn // fiddle with EOLs for target platform
|> WriteTemplate toEncoding toFile // write out in destination encoding
|> MaybeSetReadExecute // finally set permissions if asked
It processes the files one at a time, instead of in a sequence like TemplateHelper because it's too hairy to supply parallel sequences of read & write encodings. But as a bonus, it eliminates the use of mutable state I found in the sources, making it more functional, if anybody cares. The added features are the ability to supply input and output encodings, handle EOLs, and optionally set read/execute permissions on the generated file.
Here's an example of it in use:
Target "BuildLauncherScripts" (fun _ -> // Scripts really only needed for mono
let asciiEncoding = new ASCIIEncoding() // Avoid UTF-8 BOM which breaks shebang lines
let generateAScript (fromTempl : String) = // Instantiate a script from template file
let ext = fromTempl.Substring(fromTempl.LastIndexOf('.'))
processTemplate projProps // projProps to substitute template vars
asciiEncoding (srcTem @@ fromTempl) // copy from templates to build area
asciiEncoding (buildApp @@ (projProps.Item("@product")) + ext)
(ext = ".sh") true // always executable, because it's a script
generateAScript "mono-launcher.sh" // Unix script to launch mono + application
generateAScript "bat-launcher.bat" // Windows script to just launch application
) //
What I'd Like To Hear From You
- Is there a way to get
TemplateHelperto deal with input/output file encodings, EOL conventions, and setting the read/execute permissions? - If not, is the above a reasonable way of proceeding?
- In particular, is the EOL code likely to work on Windows or will it double up on returns?
- Also, is there some way to set read/execute permissions in a platform-neutral way? (My code above in
MaybeSetReadExecuteis a frighteningly kludgey mix of shelling out and imitating some .NET code I found but barely understand.)