EVOLUTION-MANAGER
Edit File: fansi.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>R: Details About Manipulation of Strings Containing Control...</title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" type="text/css" href="R.css" /> </head><body> <table width="100%" summary="page for fansi {fansi}"><tr><td>fansi {fansi}</td><td style="text-align: right;">R Documentation</td></tr></table> <h2>Details About Manipulation of Strings Containing Control Sequences</h2> <h3>Description</h3> <p>Counterparts to R string manipulation functions that account for the effects of ANSI text formatting control sequences. </p> <h3>Control Characters and Sequences</h3> <p>Control characters and sequences are non-printing inline characters that can be used to modify terminal display and behavior, for example by changing text color or cursor position. </p> <p>We will refer to ANSI control characters and sequences as "<em>Control Sequences</em>" hereafter. </p> <p>There are three types of <em>Control Sequences</em> that <code>fansi</code> can treat specially: </p> <ul> <li><p> "C0" control characters, such as tabs and carriage returns (we include delete in this set, even though technically it is not part of it). </p> </li> <li><p> Sequences starting in "ESC[", also known as ANSI CSI sequences. </p> </li> <li><p> Sequences starting in "ESC" and followed by something other than "[". </p> </li></ul> <p><em>Control Sequences</em> starting with ESC are assumed to be two characters long (including the ESC) unless they are of the CSI variety, in which case their length is computed as per the <a href="http://www.ecma-international.org/publications/standards/Ecma-048.htm">ECMA-48 specification</a>. There are non-CSI escape sequences that may be longer than two characters, but <code>fansi</code> will (incorrectly) treat them as if they were two characters long. </p> <p>In theory it is possible to encode <em>Control Sequences</em> with a single byte introducing character in the 0x40-0x5F range instead of the traditional "ESC[". Since this is rare and it conflicts with UTF-8 encoding, we do not support it. </p> <p>The special treatment of <em>Control Sequences</em> is to compute their display/character width as zero. For the SGR subset of the ANSI CSI sequences, <code>fansi</code> will also parse, interpret, and reapply the text styles they encode as needed. Whether a particular type of <em>Control Sequence</em> is treated specially can be specified via the <code>ctl</code> parameter to the <code>fansi</code> functions that have it. </p> <h3>ANSI CSI SGR Control Sequences</h3> <p><strong>NOTE</strong>: not all displays support ANSI CSI SGR sequences; run <a href="term_cap_test.html">term_cap_test</a> to see whether your display supports them. </p> <p>ANSI CSI SGR Control Sequences are the subset of CSI sequences that can be used to change text appearance (e.g. color). These sequences begin with "ESC[" and end in "m". <code>fansi</code> interprets these sequences and writes new ones to the output strings in such a way that the original formatting is preserved. In most cases this should be transparent to the user. </p> <p>Occasionally there may be mismatches between how <code>fansi</code> and a display interpret the CSI SGR sequences, which may produce display artifacts. The most likely source of artifacts are <em>Control Sequences</em> that move the cursor or change the display, or that <code>fansi</code> otherwise fails to interpret, such as: </p> <ul> <li><p> Unknown SGR substrings. </p> </li> <li><p> "C0" control characters like tabs and carriage returns. </p> </li> <li><p> Other escape sequences. </p> </li></ul> <p>Another possible source of problems is that different displays parse and interpret control sequences differently. The common CSI SGR sequences that you are likely to encounter in formatted text tend to be treated consistently, but less common ones are not. <code>fansi</code> tries to hew by the ECMA-48 specification <strong>for CSI control sequences</strong>, but not all terminals do. </p> <p>The most likely source of problems will be 24-bit CSI SGR sequences. For example, a 24-bit color sequence such as "ESC[38;2;31;42;4" is a single foreground color to a terminal that supports it, or separate foreground, background, faint, and underline specifications for one that does not. To mitigate this particular problem you can tell <code>fansi</code> what your terminal capabilities are via the <code>term.cap</code> parameter or the "fansi.term.cap" global option, although <code>fansi</code> does try to detect them by default. </p> <p><code>fansi</code> will will warn if it encounters <em>Control Sequences</em> that it cannot interpret or that might conflict with terminal capabilities. You can turn off warnings via the <code>warn</code> parameter or via the "fansi.warn" global option. </p> <p><code>fansi</code> can work around "C0" tab control characters by turning them into spaces first with <code><a href="tabs_as_spaces.html">tabs_as_spaces</a></code> or with the <code>tabs.as.spaces</code> parameter available in some of the <code>fansi</code> functions. </p> <p>We chose to interpret ANSI CSI SGR sequences because this reduces how much string transcription we need to do during string manipulation. If we do not interpret the sequences then we need to record all of them from the beginning of the string and prepend all the accumulated tags up to beginning of a substring to the substring. In many case the bulk of those accumulated tags will be irrelevant as their effects will have been superseded by subsequent tags. </p> <p><code>fansi</code> assumes that ANSI CSI SGR sequences should be interpreted in cumulative "Graphic Rendition Combination Mode". This means new SGR sequences add to rather than replace previous ones, although in some cases the effect is the same as replacement (e.g. if you have a color active and pick another one). </p> <h3>Encodings / UTF-8</h3> <p><code>fansi</code> will convert any non-ASCII strings to UTF-8 before processing them, and <code>fansi</code> functions that return strings will return them encoded in UTF-8. In some cases this will be different to what base R does. For example, <code>substr</code> re-encodes substrings to their original encoding. </p> <p>Interpretation of UTF-8 strings is intended to be consistent with base R. There are three ways things may not work out exactly as desired: </p> <ol> <li> <p><code>fansi</code>, despite its best intentions, handles a UTF-8 sequence differently to the way R does. </p> </li> <li><p> R incorrectly handles a UTF-8 sequence. </p> </li> <li><p> Your display incorrectly handles a UTF-8 sequence. </p> </li></ol> <p>These issues are most likely to occur with invalid UTF-8 sequences, combining character sequences, and emoji. For example, as of this writing R (and the OSX terminal) consider emojis to be one wide characters, when in reality they are two wide. Do not expect the <code>fansi</code> width calculations to to work correctly with strings containing emoji. </p> <p>Internally, <code>fansi</code> computes the width of every UTF-8 character sequence outside of the ASCII range using the native <code>R_nchar</code> function. This will cause such characters to be processed slower than ASCII characters. Additionally, <code>fansi</code> character width computations can differ from R width computations despite the use of <code>R_nchar</code>. <code>fansi</code> always computes width for each character individually, which assumes that the sum of the widths of each character is equal to the width of a sequence. However, it is theoretically possible for a character sequence that forms a single grapheme to break that assumption. In informal testing we have found this to be rare because in the most common multi-character graphemes the trailing characters are computed as zero width. </p> <p>As of R 3.4.0 <code>substr</code> appears to use UTF-8 character byte sizes as indicated by the leading byte, irrespective of whether the subsequent bytes lead to a valid sequence. Additionally, UTF-8 byte sequences as long as 5 or 6 bytes may be allowed, which is likely a holdover from older Unicode versions. <code>fansi</code> mimics this behavior. It is likely <code>substr</code> will start failing with invalid UTF-8 byte sequences with R 3.6.0 (as per SVN r74488). In general, you should assume that <code>fansi</code> may not replicate base R exactly when there are illegal UTF-8 sequences present. </p> <p>Our long term objective is to implement proper UTF-8 character width computations, but for simplicity and also because R and our terminal do not do it properly either we are deferring the issue for now. </p> <h3>R < 3.2.2 support</h3> <p>Nominally you can build and run this package in R versions between 3.1.0 and 3.2.1. Things should mostly work, but please be aware we do not run the test suite under versions of R less than 3.2.2. One key degraded capability is width computation of wide-display characters. Under R < 3.2.2 <code>fansi</code> will assume every character is 1 display width. Additionally, <code>fansi</code> may not always report malformed UTF-8 sequences as it usually does. One exception to this is <code><a href="nchar_ctl.html">nchar_ctl</a></code> as that is just a thin wrapper around <code><a href="../../base/html/nchar.html">base::nchar</a></code>. </p> <h3>Miscellaneous</h3> <p>The native code in this package assumes that all strings are NULL terminated and no longer than (32 bit) INT_MAX (excluding the NULL). This should be a safe assumption since the code is designed to work with STRSXPs and CHRSXPs. Behavior is undefined and probably bad if you somehow manage to provide to <code>fansi</code> strings that do not adhere to these assumptions. </p> <hr /><div style="text-align: center;">[Package <em>fansi</em> version 0.4.1 <a href="00Index.html">Index</a>]</div> </body></html>