Tools for agents: Unicode text span converter

Convert text spans between UTF-8 bytes, UTF-16 units and Unicode code points, with exact selected text.

Connect with MCP

{
  "mcpServers": {
    "super-agent-tools": {
      "url": "https://app.getsupers.com/sites/agent-text-span/mcp"
    }
  }
}
Streamable HTTP · No API key · Stateless

Use this URL in any client supporting remote Streamable HTTP MCP. Calls return structured JSON; inputs are processed without persistence.

Call convert_text_span

Offsets are zero-based and half-open: start included, end excluded. Empty spans are allowed. UTF-8 counts encoded bytes without a BOM; UTF-16 counts 16-bit units without a BOM. Both endpoints must lie on Unicode scalar boundaries; a split multibyte sequence or surrogate pair is rejected. Returned line and code-point column positions are one-based; only LF starts a new line, so CR in CRLF counts as a character on the preceding line. End position describes the exclusive endpoint. Combining marks remain separate code points: this tool does not segment grapheme clusters or count visual cursor columns. Text is never normalized, trimmed or newline-converted. Unpaired surrogate values are rejected. Useful for translating parser, JavaScript and byte-oriented tool offsets. Input text is processed without persistence.

Input and output schemas
{
  "name": "convert_text_span",
  "title": "Unicode text span converter",
  "description": "Resolve a half-open text span and return equivalent code-point, UTF-8 byte and UTF-16 code-unit offsets plus line/column positions. Reject endpoints inside encoded characters.",
  "inputSchema": {
    "properties": {
      "text": {
        "title": "Text",
        "type": "string"
      },
      "start": {
        "title": "Start",
        "type": "integer"
      },
      "end": {
        "title": "End",
        "type": "integer"
      },
      "unit": {
        "default": "code_points",
        "enum": [
          "code_points",
          "utf8_bytes",
          "utf16_units"
        ],
        "title": "Unit",
        "type": "string"
      }
    },
    "required": [
      "text",
      "start",
      "end"
    ],
    "title": "convert_text_spanArguments",
    "type": "object"
  },
  "outputSchema": {
    "additionalProperties": true,
    "title": "convert_text_spanDictOutput",
    "type": "object"
  },
  "icons": null,
  "annotations": {
    "title": null,
    "readOnlyHint": true,
    "destructiveHint": false,
    "idempotentHint": true,
    "openWorldHint": false
  },
  "meta": null,
  "execution": null
}

Run the example to see the actual result.

HTTP alternative

POST https://app.getsupers.com/sites/agent-text-span/call
Content-Type: application/json

{
  "text": "A\u00e9\ud834\udd1e\nZ",
  "start": 1,
  "end": 4,
  "unit": "utf16_units"
}

The HTTP and MCP interfaces execute the same implementation. Validation errors are returned explicitly. See the tool notes above for its supported inputs and behavior.

Resolving Cross-Encoding Text Spans

Read the explanation

In Unicode strings, characters occupy varying lengths across encodings. The musical symbol G-clef requires two UTF-16 code units or four UTF-8 bytes, but represents a single code point. Selecting a half-open span in UTF-16 units targets indices one through four. The converter scans surrogate boundaries, ensuring slices align with valid Unicode scalar values. When you execute the tool, the converter translates the span into synchronized code points, byte offsets, and one-based line and column positions without altering the raw text.

Cross-Encoding Text Span Resolution and Scalar Boundaries

How does `convert_text_span` map half-open offsets between UTF-8 bytes, UTF-16 code units, and Unicode code points?

Different runtime environments measure string indices using different units: Python and databases often track Unicode code points, JavaScript and the Java VM track 16-bit UTF-16 code units, and networking protocols or byte parsers track UTF-8 octets. The `convert_text_span` tool takes a half-open span [start, end) indexed in one of these three units, verifies that both endpoints align with complete Unicode scalar values, and computes the exact corresponding slice alongside equivalent spans in the other encodings and 1-based line/column coordinates.

The converter only recognizes newline line-breaks on line-feed ('\n', LF); any carriage return ('\r') in a CRLF pair is counted as an ordinary character on the preceding line. Offsets strictly track Unicode scalar values; combining characters, variation selectors, and multi-codepoint grapheme clusters are not grouped into visual cluster widths. Submitting an endpoint that splits a UTF-16 surrogate pair or a multi-byte UTF-8 sequence causes the tool to reject the request with a scalar boundary error.

Try a worked example

Leaving the default input in JSON arguments with text containing "A\u00e9\ud834\udd1e\nZ" (representing 'A', 'é', the musical G-clef U+1D11E, newline, and 'Z') with start=1, end=4, and unit="utf16_units", click 'Run tool'. The structured result outputs the extracted slice "é\ud834\udd1e", showing that this 3-unit UTF-16 slice translates to code-point indices 1 through 3 (length 2) and UTF-8 byte indices 1 through 7 (length 6 bytes: 2 bytes for 'é' plus 4 bytes for the G-clef).

Unicode Scalar Values versus Encoding Code Units

The Unicode Standard defines a Unicode scalar value as any Unicode code point excluding surrogate code points in the range U+D800 through U+DFFF. In UTF-16, characters outside the Basic Multilingual Plane (above U+FFFF, such as emojis and supplementary musical symbols) require a surrogate pair consisting of two 16-bit code units. The Unicode Standard, Version 15.0 – Chapter 3: Conformance

In UTF-8, characters are encoded using sequences from one to four bytes. When tools exchange string slices across language boundaries—for instance, passing JavaScript string indices to a byte-oriented network parser—slicing at an arbitrary unit boundary can split a code unit sequence and corrupt scalar values. The Unicode Standard, Version 15.0 – Chapter 3: Conformance

Offset Mapping and Boundary Enforcement

This tool constructs cumulative offset arrays for code points, UTF-8 bytes, and UTF-16 code units across the input string. It enforces that both the start and end offsets match an exact boundary of a Unicode scalar value before computing spans, preventing truncated surrogates or incomplete UTF-8 sequences. The Unicode Standard, Version 15.0 – Chapter 3: Conformance

Sources and further reading