The largest exponents we compute are on the order of 10^21 (governed by
the maximumSignificantDigits option, which has a max value of 21). That
is too large to fit into the i64 we were using when multiplying this
exponent by the value to be formatted.
Instead, split up the logic to multiply that value by this exponent
based on the value's underlying type:
Number: Do not cast the result of pow() to an i64, and perform the
follow-up multiplication with doubles.
BigInt: Do not use pow(). Instead, compute the exponent as a BigInt
from the start, then perform the follow-up multiplication with that
BigInt.
In the main spec, [[UseGrouping]] can be true or false. In V3, it may be
one of:
auto: Respect the per-locale preference for grouping.
always: Ignore per-locale preference for grouping and always insert
the grouping separator (note: true is now an alias for always).
min2: Ignore per-locale preference for grouping and only insert the
grouping separator if the primary group has at least 2 digits.
false: Ignore per-locale preference for grouping and never insert
the grouping separator.
This contains minimal changes to parse newly added and modified options
from the Intl.NumberFormat V3 proposal, while maintaining main spec
behavior in Intl.NumberFormat.prototype.format. The parsed options are
reflected only in Intl.NumberFormat.prototype.resolvedOptions and the js
REPL.
This also allows removing a bit of a BigInt hack to resolve plurality of
BigInt numbers (because the AOs used in ResolvePlural support BigInt,
wherease the naive Unicode::select_pattern_with_plurality did not).
We use cardinal form here; the number format patterns in the CLDR align
with the cardinal form of the plural rules.
Intl.NumberFormat is meant to format both Number and BigInt types. To
prepare for formatting BigInt types, this generalizes our NumberFormat
implementation to operate on Value instances rather than doubles. All
arithmetic is moved to static helpers that can now be updated with
BigInt semantics.
Other Intl objects, such as PluralRules, are to be treated as a
NumberFormat object in some AOs. There's only a handful of fields which
are to be shared between those objects - move them to a base class for
shared reuse.
This also updates the couple of NumberFormat AOs that are meant to
operate on these NumberFormat-like objects.
Alternatively, we could just have objects like PluralRules inherit from
NumberFormat directly. But that messes up the is<NumberFormat> runtime
checks, so this feels safer.
Before LibUnicode generated methods were weakly linked, we had a public
method (get_locale_currency_mapping) for retrieving currency mappings.
That method invoked one of several style-specific methods that only
existed in the generated UnicodeLocale.
One caveat of weakly linked functions is that every such function must
have a public declaration. The result is that each of those styled
methods are declared publicly, which makes the wrapper redundant
because it is just as easy to invoke the method for the desired style.
There are a few algorithms in TR-35 that need to replace digits before
returning any results to callers. For example, when formatting time zone
offsets, a string like "GMT+12:34" must have its digits replaced with
the default numbering system for the desired locale.
This is a normative change in the Intl spec:
https://github.com/tc39/ecma402/commit/f0f66cf
There are two main changes here:
1. Converting BigInt/Number objects to mathematical values.
2. A change in how ToRawPrecision computes its exponent and significant
digits.
For (1), we do not yet support BigInt number formatting, thus already
have coerced Number objects to a double. When BigInt is supported, the
number passed into these methods will likely still be a Value, thus can
be coereced then.
For (2), our implementation already returns the expected edge-case
results pointed out on the spec PR.
This is a normative change in the Intl spec:
https://github.com/tc39/ecma402/commit/f0f66cf
Our implementation is unaffected by this change. LibUnicode pre-computes
positive, negative, and signless format patterns, so we already format
negative infinity correctly. Also, the CLDR does not contain specific
locale-dependent strings for negative infinity anyways.
The general idea when ENABLE_UNICODE_DATABASE_DOWNLOAD is OFF has been
that the Intl APIs will provide obviously incorrect results, but should
not crash. This regressed a bit with NumberFormat and DateTimeFormat.
There are 443 number system objects generated, each of which held an
array of number system symbols. Of those 443 arrays, only 39 are unique.
To uniquely store these, this change moves the generated NumericSymbol
enumeration to the public LibUnicode/NumberFormat.h header with a pre-
defined set of symbols that we need. This is to ensure the generated,
unique arrays are created in a known order with known symbols. While it
is unfortunate to no longer discover these symbols at generation time,
it does allow us to ignore unwanted symbols and perform less string-to-
enumeration conversions at lookup time.
Currently, we generate separate data files for locale and number format
related tables/methods, but provide public accessors for all of the data
in one Locale.h file. Rather than continuing this trend for date-time,
relative time, etc. formatting, it's a bit easier to reason about if the
public accessors are also in separate files.
This wasn't the case for compact patterns, but unit patterns can contain
multiple (up to 2, really) identifiers that must each be recognized by
LibJS.
Each generated NumberFormat object now stores an array of identifiers
parsed. The format pattern itself is encoded with the index into this
array for that identifier, e.g. the compact format string "0K" will
become "{number}{compactIdentifier:0}".
This field is currently used to store the StringView into the compact
name/symbol in the format string. Units will need to store a similar
field, so rename the field to be more generic, and extract the parser
for it.
Instead of currency pattern lookups within select_currency_unit_pattern,
rename the method to select_pattern_with_plurality and accept any list
of patterns. This method will be needed for units.
Finding the best number format to use for compact notation involves
creating a Vector of all compact formats for the locale and looking for
the one that best matches the number's magnitude. ECMA-402 wants this
number format to be found multiple times, so cache the result for future
use.
The compact scale of each formatting rule was precomputed in commit:
be69eae651
Using the formula: compact scale = magnitude - pattern scale
This computation was off-by-one.
For example, consider the format key "10000-count-one", which maps to
"00 thousand" in en-US. What we are really after is the exponent that
best represents the string "thousand" for values greater than 10000
and less than 100000 (the next format key). We were previously doing:
log10(10000) - "00 thousand".count("0") = 2
Which clearly isn't what we want. Instead, if we do:
log10(10000) + 1 - "00 thousand".count("0") = 3
We get the correct exponent for each format key for each locale.
This commit also renames the generated variable from "compact_scale" to
"exponent" to match the terminology used in ECMA-402.
In order to implement Intl.NumberFormat.prototype.formatToParts, do not
replace {currency} keys in the format pattern before ECMA-402 tells us
to. Otherwise, the array return by formatToParts will not contain the
expected currency key.
Early replacement was done to avoid resolving the currency display more
than once, as it involves a couple of round trips to search through
LibUnicode data. So this adds a non-standard method to NumberFormat to
do this resolution and cache the result.
Another side effect of this change is that LibUnicode must replace unit
format patterns of the form "{0} {1}" during code generation. These were
previously skipped during code generation because LibJS would just
replace the keys with the currency display at runtime. But now that the
currency display injection is delayed, any {0} or {1} keys in the format
pattern will cause PartitionNumberPattern to abort.
There aren't any dangling views in as of yet, but a subsequent commit
will cause the "part" variable to be a view into an internally generated
string. Therefore, after returning from PartitionNumberPattern, that
view will be pointed at freed memory.
This commit is to set the precendence of not returning a view to "part".
Currencies are a bit strange; the layout of currency data in the CLDR is
not particularly compatible with what ECMA-402 expects. For example, the
currency format in the "en" and "ar" locales for the Latin script are:
en: "¤#,##0.00"
ar: "¤\u00A0#,##0.00"
Note how the "ar" locale has a non-breaking space after the currency
symbol (¤), but "en" does not. This does not mean that this space will
appear in the "ar"-formatted string, nor does it mean that a space won't
appear in the "en"-formatted string. This is a runtime decision based on
the currency display chosen by the user ("$" vs. "USD" vs. "US dollar")
and other rules in the Unicode TR-35 spec.
ECMA-402 shies away from the nuances here with "implementation-defined"
steps. LibUnicode will store the data parsed from the CLDR however it is
presented; making decisions about spacing, etc. will occur at runtime
based on user input.
There is quite a lot to be done here so this is just a first pass at
number formatting. Decimal and percent formatting are mostly working,
but only for standard and compact notation (engineering and scientific
notation are not implemented here). Currency formatting is parsed, but
there is more work to be done to handle e.g. using symbols instead of
currency codes ("$" instead of "USD"), and putting spaces around the
currency symbol ("USD 2.00" instead of "USD2.00").
