<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8">
<style>
div.markdown { white-space: normal; }
div.plaintext { white-space: normal; }
body { font-family: sans-serif; }
h1 { font-size: 1.4em; }
h2 { font-size: 1.2em; }
h3 { font-size: 1.1em; }
blockquote { margin: 0 0 5px; padding-left: 5px; border-left: 2px solid #777777; color: #777777; }
blockquote blockquote { border-left-color: #999999; color: #999999; }
blockquote blockquote blockquote { border-left-color: #BBBBBB; color: #BBBBBB; }
a { color: #3983C4; }
blockquote a { color: #777777; }
blockquote blockquote a { color: #999999; }
blockquote blockquote blockquote a { color: #BBBBBB; }
code { margin: 0; padding: 0 0.4em; border-radius: 3px; background-color: #F7F7F7; }
pre > code { padding: 0px; }
math[display="inline"] > mrow { padding:5px; }
div.footnotes li p { margin: 0.2em 0; }
</style>
</head>
<body>
<div class="markdown">
<p dir="auto">Ok, here’s what’s going on:</p>
<ol>
<li value=1><p dir="auto">The message is encoded in windows-1252, and contains non-breaking spaces (encoded as the byte 0xA0).</p></li>
<li value=2><p dir="auto">Your terminal is using a different character encoding (probably UTF-8), in which 0xA0 (as used) does not map to a character. (In UTF-8, 0xA0 is a “continuation character”, which is only valid as a non-first byte in a multi-byte sequence.) In the absence of a valid UTF-8 character code, the terminal gives up and displays the value of the unmappable byte in angle brackets. However, “<A0>” is only how the terminal displays the nbsp character. The .eml file itself does not contain the four-character string “<A0>”.</p></li>
</ol>
<p dir="auto">Dealing with non-breaking spaces can be confusing, since they are difficult to differentiate from normal spaces in many editors, and they have different character codes in the common 8-bit encodings and UTF-8. But they are usually encoded either as the single byte 0xA0 (in the 8-bit encodings) or as the two-byte sequence 0xC2A0 (in UTF-8). As others have pointed out, a quick call to <code>tr '\240' ' '</code> to translate the nbsps to normal spaces will often do the trick. If you happen to be using perl, <code>use feature “unicode_strings”</code> will make pattern-matching behave properly with nbsps (yes, even with strings from windows-1252-encoded files!) — for example, it will make <code>\s</code> match nbsps, which it normally doesn’t.</p>
<p dir="auto">Best of luck with the scripting!</p>
<p dir="auto">Galen</p>
<p dir="auto">On 6 Apr 2019, at 4:27, Randy Bush wrote:</p>
</div>
<div class="plaintext"><blockquote><p dir="auto">i receive an email<br>
<br>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:52.0)<br>
Gecko/20100101 PostboxApp/6.1.13<br>
MIME-Version: 1.0<br>
Content-Type: text/plain; charset=windows-1252; format=flowed<br>
Content-Transfer-Encoding: 8bit<br>
Content-Language: en-US<br>
<br>
the text has funny space characters that i see if i save the text to<br>
disk and look at it with less<br>
<br>
<A0>0.<A0><A0> flo....: 2.31 2018.11.03<br>
<br>
<A0><A0><A0><A0><A0><A0><A0><A0><A0><A0> 1.<A0><A0> CLIMATE ACTION<br>
<A0><A0><A0><A0><A0><A0><A0><A0><A0><A0><A0><A0><A0> * (N)ew (M)odify (D)elete..: N<br>
<br>
<A0><A0><A0><A0><A0><A0><A0><A0><A0><A0> 2. * NAME OF CLOUD: cumulus<br>
<br>
i presume the sender is thunderbird and they have created the text with<br>
some sort of windows encoding on a mac?<br>
<br>
how can i save the content as vanilla ascii text?<br>
<br>
randy<br>
_______________________________________________<br>
mailmate mailing list<br>
mailmate@lists.freron.com<br>
<a href="https://lists.freron.com/listinfo/mailmate">https://lists.freron.com/listinfo/mailmate</a></p>
</blockquote></div>
<div class="markdown">
</div>
</body>
</html>