Blog

2021/10/02

Support for Unicode

Another new feature is to support unicode (UTF-8) text format. You can now read and write unicode characters from source code (C++, Java), or from scripts. Non-ASCII characters can be scripted like this:

text s: "Hello \xD1\xBEorld!"    // equals "Hello Ѿorld!"

Here \x is followed by hexadecimal code of the 8-bit character sequence. Codes D1 and BE ("\xD1\xBE") combined are the UTF-8 code for the letter Ѿ. See the reference page for more information about escape (backslash) sequences.

The unicode support was made possible by a big refactoring under the hood. I started using own string type instead of native string. Meanscript’s string type (MSText) uses internally an integer array to save text in UTF-8 format. This helps also to minimize dependencies to external libraries. The plan is to use native strings (e.g. java.lang.String) only in public interfaces (API).

More examples at Meanscript's GitHub page.

2021/09/07

New 64-bit data types

Meanscript has two new data types: 64-bit integer and floating-point number, in addition to old 32-bit ones (all data types). 32-bit integer is the basic data unit in Meanscript, but especially for floating-point numbers 64-bit presentation is more common to have more accuracy.

You can define the 64-bit numbers in a script

int64 bigInteger: 1234567891234; // 64-bit integer
float64 bigFloat: 12.123456789;  // 64-bit floating-point number

or source code

MSBuilder builder ("example");
builder.addInt64("bigInteger", 1234567891234);
builder.addFloat64("bigInteger", 12.123456789);

Variables in Meanscript are stored in a 32-bit integer array, so floating-point numbers must be converted to integers to store them in Meanscript. For 64-bits, the conversion is wrapped inside a macro FLOAT64_TO_INT64_FORMAT(f) to make implementation for each supported language. In C++ the conversion can be done efficiently with some casting trickery:

#define FLOAT64_TO_INT64_FORMAT(f) ((int64_t&)(*(&f)))

In Java there’s a function for that in Double library

#define FLOAT64_TO_INT64_FORMAT(f) Double.longBitsToDouble(f)

After the conversion, the data can be stored to a 32-bit integer array by splitting 64-bit integer in half.

Conversions from integer to floating-point and back can vary according to environment, so there’s a unit tests that make sure the conversions are identical in all environments, e.g.

FLOAT64 f64 = -12.123456789;
INT64 longBits = FLOAT64_TO_INT64_FORMAT(f64);
TEST(longBits == -4600357519365344569l);

For more information check out GitHub's reference page.

2021/08/25

Welcome to my new website!

I’ve been building a new web site for awhile and learning some NodeJS in the process. Like Meansript, I’ve tried to keep the code as lightweight as possible, with minimal dependencies to external libraries. In the core of the website code is a system that caches all the web site content, so there virtually no file reads at all while running it. There’s also a tracking system that uses HTML storage for session ID, instead of cookies. This blog and multi-page articles with indexing are defined in JSON files. All this is implemented with about 25 KB of JS code.

In the future I try to make the code more robust and improve reporting, in addition to some UI issues. Maybe some day I make the code open source after I’ve cleaned up stuff that’s hard-coded for this web site!

↑ top