Xah Lee, 2011-02-05, …, 2011-12-06
Vast majority of computer languages use ASCII as its character set. This means, it jams multitude of operators into about 20 symbols. Often, a symbol has multiple meanings depending on contex. Also, a sequence of chars are used as a single symbol as a workaround for lack of symbols. Even for languages that use Unicode as its char set (e.g. Java, XML), often still use the ~20 ASCII symbols for all its operators. The only exceptions i know of are Mathematica, Fortress, APL. This page gives some examples of problems created by symbol congestion.
Here are some common examples of a symbol that has multiple meanings depending on context:
In Java, [ ] is a delimiter for array, also a delimiter for getting a element of array, also as part of the syntax for declaring a array type.
In Java and many other langs, ( ) is used for expression
grouping, also as delimiter for arguments of a function call, also as
delimiters for parameters of a function's declaration.
In {C, Perl} and many other langs, : is used as a separator in a ternary expression (e.g. (test ? "yes" : "no")), also as a namespace separator (e.g. use Data::Dumper;).
In URL, / is used as path separators, but also as indicator of protocol. e.g. http://example.org/comp/unicode.html
In Python and many others, < is used for “less than” boolean operator, but also as a alignment flag in its “format” method, also as a delimiter of named group in regex, and also as part of char in other operators that are made of 2 chars, e.g.:
<<
<=
<<=
<>.
The above are just some examples to illustrate the issue. There are perhaps 100 times more.
Here are some common examples of operators that are made of multiple characters: || && == <= != ** =+ =* := ++ -- :: // /*.
The tradition of sticking to the 95 chars in ASCII of 1960s is extremely limiting. It creates complex problems manifested in:
String Escape mechanism, for example, C's backslash {\n, \r, \t, \/, …}, widely adopted. A better solution would be Unicode symbols for unprintable chars. Example:
See also: Computing Symbols in Unicode.
The backslash string escape mechanism directly leads to crazy leaning toothpicks syndrome, especially bad in emacs regex. e.g.:
"<img src=\"\\([^\"]+\\)\" alt=\"\\([^\"]+\\)\" width=\"\\([0-9]+\\)\" height=\"\\([0-9]+\\)\">"
This is particularly bad in regex. For example, ^ has multiple meanings depending on where it is placed. If in the beginning, it's a line beginning marker, if as first char inside square brackets e.g. [^…] then it's a negation, otherwise it's literal.
Many other regex chars also has special meaning, some depends on their placement. e.g.:
^ $ ? | . + \ - { } ( ) [ ] ….
Whether a symbol's meaning is literal, or whether their position changes meaning, is completely ad hoc.
The lack of bracketing symbols leads to varieties of unnecessarily complex string delimiters to help solve the problem of quoting.
Python's triple quotes: {'''a''', """a"""}. (See: Strings in Perl and Python.)
Perl's varying delimiters: {q(), q[], q{}, m//}.
Perl, PHP, unix shell's heredoc. (See: PHP: String Syntax and Heredoc.)
(See also: Computer Language Design: String Syntax ◇ HTML6: Your JSON and SXML Simplified.)
See: Emacs's Key Notations Explained (/r, ^M, C-m, RET, <return>, M-, meta).
HTML entities, e.g. { &, <, >, ", α, α, α, …}.
The HTML entities are invented partly as a mechanism of avoiding symbol jam of these characters: & < >, and partly as a kludge to overcome the difficulty of inputting Unicode symbols, e.g. © ™ α → …, and partly as a kludge to avoid char encoding and transmission problem (i.e. at the time, only ASCII is widely recognized.). (See: HTML/XML Entities (Character/Unicode/Symbol) List.)
URL percent encoding and encoding Unicode in URL, e.g. http://en.wikipedia.org/wiki/St._Jerome_in_His_Study_%28D%C3%BCrer%29. (See: URL Percent Encoding problems.)
The complexity in resolving the ambiguity of the Ampersand char in URL and CGI protocol. See: URL Percent Encoding and Ampersand Char.
Javascript Encode URL, Escape String
All these problems occur because we are jamming so many meanings into about 20 symbols in ASCII.
The language designer Guy Steele recently gave a very interesting talk. See: Guy Steele on Parallel Programing. In it, he showed code snippets of his language Fortress, which freely uses Unicode as operators.
For example, list delimiters are not the typical curly bracket {1,2,3} or square bracket [1,2,3], but the Unicode angle bracket ⟨1,2,3⟩. (See: Matching Brackets in Unicode.) It also uses the circle plus “⊕” as operator. (See: Math Symbols in Unicode.)
Most of today's languages do not support Unicode in function or variable names, so you can forget about using Unicode in variable names (e.g. α = 3) or function names (e.g. “lambda” as “λ” or “function” as “ƒ”), or defining your own operators (e.g. “⊕”).
However, there are a few languages i know that do support Unicode in function or variable names. Some of these allow you to define your own operators. However, they may not allow Unicode for the operator symbol. See: Unicode Support in Ruby, Perl, Python, javascript, Java, Emacs Lisp, Mathematica.
Today, it's trivial to create a keyboard layout to type any set of Unicode symbols you choose. See: How to Create a APL or Math Symbols Keyboard Layout.