Text modified is NOT a Text anymore

A Catch-22 Problem

I've coded my way into a problem - it's a classic "catch-22" - the solution to the problem causes yet another problem, and we recurse.

I've never paid much attention to the Swift Text after one applies a few text modifications - it all just works. And that "just works" is priceless. Steve Jobs should have trademarked that phrase. It is one of the attributes of developing in the Apple ecosystem that is a joy and magic!

Here's my first problem: I'm developing a calculator app for a non-english culture - it does not use the typical alphabet to describe the numbers. Well that's an easy problem in iOS and the touch screen interface. It is a problem that occured to me in early 2000. Designing products for non-English humans was expensive. Each language needed it's own keyboard, or a trick to remap the QWERT keyboard of English to something different. I was well aware of the Dvorak keyboard but take that concept exponential and one gets into prohibitive cost for product design.

Steve's new touch screen interface, in 2007, may not have been first - but it was the best and set off a new wave of innovation. I recognized the genius of this device, went out and bought the first model and have been a loyal fan ever since. With the release of Swift / SwiftUI the ability to develop in the ecosystem became very attractive (I hated Objective-C).

Ok so designing a product for a non-standard English speaking audence should be a great learning experience... so I set off to do this experience. The iOS ecosystem would provide a readily adaptive system to deal with the language problems - and I'd learn a lot of unknown techniques.

A new set of Glyphs

The challenge of a non-standard set of glyphs is easily solved by the SF Symbols tools and techniques all rolled into Xcode and the ecosystem. In fact the new features in iOS-17 make it very easy to use the ImageResource in my code.

That ImageResource becomes an Image and one can place an Image inline with other Text using interpolation. So the use of a Kaktovik numeral twelve becomes code such as:

Text("\(Kaktovik_12)")

Easy peasy! Now that only works for the numbers 0-19 in the Kaktovik Inupiaq language, which uses a base-20 number system. Lucky for me they also use a place notation in their number system. The invention of the positional notational system was a momental human achievement, some time before history began.

Concatenation of Glyphs

To make a word, we join together letters. To make a sentence we join together words. This process of joining and the rules that apply (like white space between words) is generally called concatenation in the computer field. Add reading code that does concatenation and Maths can get complex. Because we developers have found ways to overload the meaning of symbols like the plus sign.

When we have learned to read and write in our language - there are so many little neuance we ignore consciously. So take the easy maths equations:

12 + 18 = 30

That equation is made up of twelve symbols (I'm counting the white-space as a symbol) concated together into a sentence style meaning of mathematics.

Swift's Text object has a function built-into it that allows concatenation via the overloaded + (plus) sign operation. So I could easily add the Text strings together to form the mathematical sentence in english. And I can do similar in Inupiaq using the Kaktovik numerals.

12.0 + 18 = 30.0 in Kaktovik numerals (base-20)

Where is the essence?

When I abstract the glyphs of an alphbet and it's number system - I have to have somewhere to store or keep the essence of the character. But where is that kept in the computer? The letter "G" has a meaning - where is this meaning stored inside the vast computational machinery of the computer?

As it turns out - I do not believe the G-ness is embedded in the computer. It is only within the human. And this dislocation of meaning is very interesting. Now let's do the same for the numeral "8". Where is the meaning stored? Ah-ha, it is stored in the computer! The computer knows that the 8-ness of the glyph and can apply the 8-ness (it's value) to mathemtatical operations (such as plus, and multiplication).

Is it true - that numerals essence, their value is stored in the computer, yet the essence of a letter is not stored in the computer?

I'm yet uncertain of this... But I do know I cannot treat my Kaktovik numeral glyphs as a number. But can treat them as a letter. This is starting to bother me now that I've discovered the distinction. I may be barking up the tree that Gottlob Frege chopped down to publish his "On Sense and Reference" book. The identity of the "G" is stored in the alphabet, and in the font, and in all the books on the computer. Yet the sense of the "G" is in the mind of the human upon each occurance. However, the "8" at first glance appears similiar, although the computer may peform mathematic operations with the "8". To do these mathematical functions the sense of the "8" must be stored within the computer - but, where? The alphabet and the charicture set have no more properties for the "8" than for the "G" - does it? Perhaps one would consider the order of glyphs, the sense one can obtain for the ordering of the numerials is much richer than for the ordering of the alphabet. But this ordering does not produce the mathematical functions that are inherently done with numbers.

"The reference of a sentence is its truth value, whereas its sense is the thought that it expresses."
-- Gottlob Frege

Gottlob Frege - On Sense and Reference

10.0 + 8 = 18.0

Adding Glyphs

Getting back to our main story... The computer stores the letters and numbers in some way (unconcerning for us now) - yet the sense of those glyphs is also stored and represented in the computer. The fact that the glyphs are sectioned off into various parts of the character set is also meaningful. Yet withing the Apple domain I can add to the letter type glyphs, but not the number type glyphs - interesting.

I say this because I can use one of my Kaktovik numerial images in the calculator program I'm developing and then "inject" the symbol into the Swift Text object just as one would add any character in the alphabet. Well almost the same - but it has similiar properties once injected.

Text("\(kaktovik_8)")

That syntax is Swift's language syntax meaning - treat this set of characters, inside the quotes, as one chunk of text to display upon the screen. This inner escaped parenthetical is the name of an image in the Asset catalog - which happens to represent the numerial 8. This asset, the image of the kaktovik "8" now is treated like any other character in the alphabet - yet it is not in any way treated like a number. The computer cannot do math with this kaktovik 8. Programming that magic will take lots of work.

... up until now ...

I have even written lots of base-20 mathematics for my calculator app and for the most part it's all working as expected. The catch... I wanted to write test for the calculator functionality, because - well...; developer's are renown for thinking they have told the computer exactly how to do some small detail - and just being flat out WRONG. So writing test for basic math in base-20 with a set of characters that I have to really focus upon to check the math is tedious - the perfect job for a computer. Apple has created a couple of really great frameworks to make testing your App very easy - compared to the old days... XCTest contains the core functionality to write and execute unit test as well as the UI test.

So I write a few dozen unit test - all is going well. I have encluded the ability for my calculator to do base-10 (decimal) math, as well as the base-20 (vigesimal) math. It helps to just have the calculator do the math both ways so that the punny human can compare answers easily.

The next step - let's write some UI Test for the calculator to double check that the display has the correct characters (kaktovik numerals). To do this we need those images to hold some "real values" that are testable. One easy way will be to associate the base-10 number with the Text object for each symbol. Using the XCTest frameworks accessability info to check the symbol's identifier. That is as easy as:

Text("some concept")
.accessabilityIdentifer("beautiful")
.accessabilityLabel("mind")

And that works really well inside a test assertion statement:

let oneSymbol = app.staticTexts["beautiful"]
XCTAssertEqual(oneSymbol.label, "mind")

Addition and Subtraction of Number Bases - Video

ModifiedContent<>

Here's the problem... finally. When you add a modifier to the Text object it returns as a ModifiedContent object. Which the rest of the Swift domain seems just fine with. But one feature of the Text object is that concatenation ability. The ModifiedContent object lacks that superpower. So my use of Text concatenation breaks if I add testability identifier (or VoiceOver hints and abilities) ... OUCH!

Cannot convert return expression of type 'ModifiedContent<Text, AccessibilityAttachmentModifier>' to return type 'Text'

What does one have to do to get the Text back and still use the modifiers?

The Catch-22: If I use the modifiers to Text - I cannot use the Text's concat feature.

I really need the ability to stick two or more symbols together (read - the perfect job for concatenation). I also want to test my App but the recommended identifier to locate the symbol in the View hierarchy is the .accessabilityIdentifer and that modification mutates the Text into a ModifiedContent (without the ability to concatenate).

Well, some fooling around and these are the things I've tried...
- Type casting the ModifiedContent as Any as! Text
- searching StackOverflow and other sites for "ModifiedContent"
- going on long walks with my debugging dog, Malibu
- shaking my fist in the general direction of Cupertino

If you have better ideas I'd love to try them out...

A Solution

It took me a few days of procrasting before I could let go of my awesome idea of injecting the Kaktovik images into the Text SwiftUI component. Since this was my first awesome idea it took quite a while to - make a bridge - and get over it.

I made a simple example App so I could study the problem (it's on my GitHub - TestAccessibilityText). In this simple App I could reproduce the problems with injecting accessibility info to the Text components. Which then causes compile errors when applied to the Text objects.

The solution was much easier than some of my attempts to cast the ModifiedContent object back into a Text (that didn't work). And I read up on type erasure and opaque types as well as other techniques. But when I went back to the basic in my simple example app it all worked out. The solution was putting the Kaktovik symbols (Image type) into a simple Array. Then instead of using the Text object inside a View... I have to write a bit of iteration code over the array to put the images on the screen.

Precision

The simplest calculator have only 4 functions (+ - x / ). Yet they are doing many more functions than it appears. To test this out, type in this expression and see what value is returned in your calculator of choice.

12.5 x 2 = ?

Then try this one: 12.5 x 2.000001 = ?

Did you notice a difference - did you care about precision? Have you noticed that your calculator is doing precision manipulation under-the-covers. Try:

1 / 10 = ?
So many calculators are performing some result manipulation to give humans the answers we expect. The binary representation of 1/10 is not a simple 0.1 as your calculator will display. So how is it doing this manipulation?
Did you know that the computer science problem of printing out floating point numbers was not well know until... 1990 WOW!

on 2nd thought...

Little did I know when I spent a day or two reworking the existing design decision of using Text components for the storage and display of SF-Symbols into a design of using a simple Array as storage and then iterating over the Array to put the symbols on the display that it would cost so much. It required lots of reworking of existing almost functionally correct code. Lots of time to change so many slightly related functions. But what cost the design most... the loss of the Text and its awesomeness of Dynamic fonts.
I didn't even think about it in the first place, just implemented the ability to resize the characters and to scale down when the line length got high. This new Array has none of Text's abilities. So I got a few new issues... clipped character symbols and truncated equations on the display. I'm not at all happy about this revelation now that I see it.

Wow! What a great first - obvious decision to use Text.

Now for the work to move back to using both Array & Text. I'm thinking Array for storage and Text for display. I'm hoping that will work around the problems of accumulating characters into the Text component as storage - allowing me to write and use UI Testing.

References & Notes

Rounding Error: https://www.cs.drexel.edu/~popyack/Courses/CSP/Fa17/extras/Rounding/index.html

Printing Floating-Point Numbers: https://www.ryanjuckett.com/printing-floating-point-numbers/

A "Catch-22" is "a problem for which the only solution is denied by a circumstance inherent in the problem or by a rule." For example, losing something is typically a conventional problem; to solve it, one looks for the lost item until one finds it. But if the thing lost is one's glasses, one cannot see to look for them – a Catch-22. The term "Catch-22" is also used more broadly to mean a tricky problem or a no-win or absurd situation. Catch-22 is a satirical war novel by American author Joseph Heller. He began writing it in 1953; the novel was first published in 1961.

Page updated

Google Sites

Report abuse