Typing around the world

While on the topic of AddLanguageProfile - Vista changes the behaviour of this function.  In Windows XP, it would install the TIP across the user's currently installed languages, and just to be helpful, add Chinese (Simplified and Traditional), Japanese, and Korean languages with the TIP installed for each of those languages.  That's not too hard to fix up.  However in Vista, this simple function call will not only , but will also add every single language installed on the system to the user's profile - resulting in the following impressive dialog:

The list of languages goes on seemingly for ever.  Not only does this list every language - it also lists every sublanguage - Arabic (Saudi Arabia), Arabic (Iraq), Arabic (Egypt) and so on and so forth.  For good measure, the languages aren't installed alphabetically, but rather in the order of the language identifiers (0x0401 for Arabic (Saudi Arabia), 0x0402 for Bulgarian)...

Ctrl+Space, Alt+Left Shift, Ctrl+Shift and other Windows language hotkey frustrations

Some time ago, Michael Kaplan wrote about the Ctrl+Space hotkey and its broken configuration in Windows Vista.

What is particularly evil about the Ctrl+Space combination is that takes control even when the language is not selected.  So you can be happily typing away in US (English) or German (Austria) or whatever language takes your fancy, decide you want to select the current column in Excel (Ctrl+Space) and you instead will be thrown into the first available IME language.

It can even happen if the IME is not even installed.  Here's the scenario: if you have a text input processor that is registered for all languages (i.e. call  ItfInputProcessorProfiles::AddLanguageProfile with langid = -1), then Windows will behave as if the IME is installed for CJK languages, and list it in the language bar, and enable Ctrl+Space -- but without listing it in the Text Services and Input Languages dialog, and without listing the Ctrl+Space hotkey in Advanced Key Settings.  The only way to remove these bogus languages from the language bar, and disable Ctrl+Space, appears to be to remove the TIP.  Even after that, you will probably find that you have to restart Windows in order for the Language Bar to figure out which languages are actually installed...

Things are just as bad with Alt+Left Shift - the key combination that anybody who works in more than one language (and many who don't) will no doubt have experienced to their dismay...  Some of the side effects that Alt+Left Shift causes:

• Hotkeys that use Alt+Shift+key will switch the input language as well as firing
• Alt+Tab and Alt+Shift+Tab can switch languages in some situations
• This hotkey is accidentally pressed so frequently that someone even created a tool to fixup text typed with the wrong keyboard!
• If you uninstall all languages/keyboards/etc bar 1, Apply, and then add another language, the settings are reset to the defaults.  This one seems to have been fixed on Vista.
• Dizziness may occur.

Ctrl+Shift (default behaviour - switch between keyboards for selected language) has the additional behaviour of switching text direction.  So, if you have 2 keyboards installed for a language, and press Right Ctrl+Right Shift to switch to Right-to-left in a text control, you will then have to press Ctrl+Shift AGAIN to switch back to the previous keyboard.

The grave accent [`] was added as a keyboard switch hotkey in Vista.  Once you select this hotkey, there does not appear to be any way to type a grave accent...

The next build (7.0.241.0 or later) of Keyman Desktop (Professional) will add the ability to set your own hotkeys for all installed languages...  This new version should be available in a week or two.

Managing customer licences in the Keyman Developer CRM

I have just updated the documentation around the Keyman Developer CRM - specifically, details on how to manage your customer licence records and create new customer licences for your custom Keyman-based products.  See http://tavultesoft.com/keymandev/documentation/70/tutorial_crm.html for details!

A case study on why you should NEVER, EVER check in a last minute fix

Yesterday we released Keyman 7!  The excitement may have been somewhat overshadowed by a rather larger product release on the same day in the US, and ironically, an update to MSKLC was released on the same day.

I'm not writing about the new features etc etc - if you want to read about those, visit our website. 
I'm writing about the bugs.  Perhaps this is airing the dirty laundry but here goes anyway.

The day before release, our testers identified a couple of issues.

 

The first of these was purely cosmetic.  If a user had too many languages installed, the languages tab would not fill the available space but would leave about 100 pixels of whitespace at the bottom - see image.  This space was there because the User Interface Language button was moved from there to the Options page.

But for some silly reason, we decided to put in a fix.  So we put in a fix, tested it (well, obviously we didn't...  wait and see) and put the release online with only one minor issue - the Configuration dialog now did not work at *all*:

Fortunately, that was only online for a couple of minutes before we took it down - and rolled the changes back.  The second issue was more significant.  Pressing ENTER in the Activation dialog (as opposed to clicking the Activate button with the mouse) would open a Windows Explorer window instead of processing the activation:

We quickly put in a fix for this - and broke the Activate button.

Last minute fixes:

• Are done in a reckless state of mind brought on by the euphoria of impending release
• Never get tested properly
• Do not get proper attention to design and potential side-effects

The moral of the story is: NEVER, EVER do a last minute fix.  If a fix is important enough to do at the last minute, it is important enough to delay the release by a full test cycle...

Translating the Keyman Desktop UI - again!

I was asked via email this morning about the changing locale.xml file format between build 223 and build 224 of Keyman Desktop.  I had meant to write an update on this blog for my previous post about the file format - so here it is.

We reworked the locale.xml file format in build 224 to make it more consistent and eliminate some issues with extensibility.  I believe the new format is more logical - here's an overview:

<?xml version="1.0" encoding="utf-8"?>
<Locale>
  <String Type="PlainText" Id="S_ShortProductName">Keyman Desktop</String>
</Locale>
<Dialog Id="DownloadKeyboard" Width="600" Height="400">
  <footerheight>40</footerheight>
</Dialog>

Each string has a type of PlainText, HTML, or FormatString, according to how it is used.  PlainText cannot contain markup of any kind; HTML can.  FormatStrings are used mostly in dialog boxes and other messages, and cannot contain markup, but often contain parameters such as %0:s or %1:d.  These parameters can be put in any order in the output string - and can even be repeated.

We also have some new information - each HTML-based dialog in Keyman Desktop Configuration has an entry in the file that specifies its pixel width and height, and optionally some internal dimensions as well.  In the example above, the footerheight for the DownloadKeyboard dialog is given as 40 pixels.

That's all there is to the format...  Of course the new file format is wonderful and is going to solve the world's problems, but for anyone who already did a translation, that's not going to make them happy!  (Vincent?)

So, I wrote an XSLT template to translate the build 223 format to build 224 format.  The template and some wrappers to execute it are available at: www.tavultesoft.com/support/files/.  There are two wrappers for the template (or you can use your own favourite XSLT processor):
* An IE only web page - convertlocale.html  (uses ActiveX to access Msxml2.DomDocument)
* A Windows Scripting file - convertlocale.js

To run convertlocale.js, start a command prompt and type:

cscript [/nologo] convertlocale.js <infile.xml> [<outfile.xml>]

If no outfile is given, then the output is written to stdout.

If you have any issues with the conversion, please let me know.

Translating the Keyman Desktop UI

The Keyman Desktop 7.0 Beta 2 UI is now both customizable and localizable.  Localization of the user interface is possible via editing a single XML file called locale.xml.

The default locale.xml file is in the folder

[Program Files]\Tavultesoft\Keyman Desktop Professional 7.0\xml\

First steps to creating a translation

1. Find the language code for your translation
   

   Use the ISO 639-2 or ISO 639-3 language code (we recommend using the ISO 639-3 codes as many Keyman keyboards support languages not included in ISO 639-2!).

2. Create a folder for the translated locale.xml file and copy the default locale.xml into it
   
   The folder should be called:
   
     [Program Files]\Tavultesoft\Keyman Desktop Professional 7.0\locale\langcode

File Format

The file format is straightforward - most strings have identifiers (e.g. SKButtonOK for the OK button).  The identifier of the string is the XML tag name.  We have three different types of strings in the file - plain, HTML and format strings.

• Format strings often include %#:s or %#:d markers that are placeholders for parameters, and should be included in a translated string.  The order of the markers in the translated string is not important.
• Plain strings must be just that - plain text.
• HTML strings can include tags - there are only a few of these, and they are flagged in the locale.xml file.  The most obvious of these is in the Welcome screen text.

The only exception is the strings on the tray menu, which do not have identifiers, but instead use the following format (note that the Caption attribute should not include the &amp; entity):

    <Menu Caption="Keyman Configuration...">localized &amp;text...</Menu>

The &amp; entity is used to mark hotkeys in menu items, labels and buttons.  It can be used only with strings identified as formatted messages (these mostly start with SK) and with the Menu strings.

Editing the File

First, edit the file's language information - SKUILanguageName, SKUILanguageNameWithEnglish, and SKLanguageCode.

<!--   The name of the UI language, localized -->
<SKUILanguageName>English</SKUILanguageName>

<!--   Show the name of the UI language, localised, with English name in parentheses (this message is used when the user gets stuck in a strange UI language) -->
<SKUILanguageNameWithEnglish>English</SKUILanguageNameWithEnglish>

<!--   The language code for the current translation -->
<SKLanguageCode>en</SKLanguageCode>

Note that any entries missing from the translated locale.xml will be retrieved from the default file.

Editing other Files

If you find that you need to do more work on the user interface, you can always copy the other files from the default xml folder into an xml subfolder of your locale folder, and edit them directly.  Note that the XSL file for each dialog (e.g. activate.xsl, or keyman.xsl) references other files - these other files must also be copied to the locale xml folder:

  [Program Files]\Tavultesoft\Keyman Desktop Professional 7.0\locale\langcode\xml

The format of these files is somewhat more complex; I may talk about them in the future.  But feel free to play around and edit!

Finally...

The Keyman Developer user interface is also localizable (not 100% ready yet), but it primarily uses gettext, with a small amount of translatable XSLT.  We will include the gettext .pot template in the release of Keyman Developer.

The usual beta warning applies - these file formats are not necessarily 100% stable, and some changes may occur.  However, we expect only that more strings will be added to the final locale.xml, and we don't expect structural changes.

I know I said I was going to write about the product management website - that will be coming soon.  While reviewing the current functionality of the product management website (and I was actually getting ready for the blog post) we discovered a couple of issues, which I need to correct before trying to take you through the process.

So ... instead I am writing about Keyman 7 Beta 1.1.  I don't want to call it Beta 2 - we haven't got all the major issues corrected yet - we haven't even started addressing most of the issues reported!  But there have been a lot of changes over the last 2 and a half weeks (is that all?!?) and we'd like to get them out for review and testing as soon as possible.

I am quite tempted to put out a weekly build - except that the process of:

1. the usual commit/build/test process (which we run multiple times a week),
2. uploading to the website, and
3. writing about the changes

just takes up too much brain space and probably pushes out the release date each time we do it.  So we'll release updates as frequently as seems necessary...  Which is pretty obvious, really.

In the Beta 1.1 build, we have corrected/completed approximately 150 issues/featurepoints (50 of which were listed on the online beta issues list).  Major work has mostly been done in Keyman Developer, and focused around the user experience in the keyboard editor, package editor, branding editor and distribution editor.  We are also about 75% through converting the Keyman Developer user interface to 100% Unicode (previous versions of Keyman Developer ran on Windows 9x and only had selected Unicode components).

We have a few more build/test cycles to go through before Beta 1.1 is ready, and then we'll upload it.  While you are waiting, here is a screenshot of the updated Keyboard Editor to whet your appetite (font smoothing/ClearType is unfortunately switched off in this screenshot as I am grabbing it over Remote Desktop tonight):

There are still some issues in this screenshot - the "Font is Code2000" text; the preview key display is messed up; the watermark is overwhelming... but you get the idea.  However, the most important feature is the inclusion of the Source tab - you can see all the changes you make to the keyboard in real time and switch back and forth between the source and gui editors instantly.

How to create a shareware product in Keyman Developer 7 Beta 1

I am going to take you through the steps to create a shareware product in Keyman Developer 7 Beta 1.  Much of the user interface will change with Beta 2 and no doubt some of the steps will be restructured somewhat, but the concepts should remain fairly similar.

Before you start, make sure you have a keyboard, and a package with documentation and fonts.  I won't cover any of that in this example.  I am assuming that you are familiar with Keyman Developer 6 and Keyman 6.

A few things have changed from Keyman 6 with how Keyman is distributed.  Keyman has been split into two components:

• Keyman Engine - installed in C:\Program Files\Common Files\Tavultesoft\Keyman Engine 7.0
• Keyman Desktop (formerly Keyman Standard Edition) - installed in C:\Program Files\Tavultesoft\Keyman 7.0

The Keyman Engine includes:

• keyman32.dll - The Keyman runtime keystroke processor
• kmcomapi.dll - Keyman's COM API interfaces
• keyman.exe - The tray icon application and background program.

Keyman Desktop includes:

• kmshell.exe - Keyman Configuration, Splash dialog and related
• keyman.pxx - Keyman product description file
• Help files and related files

The idea is that you can replace the files in Keyman Desktop with your own product files.  You can choose to keep the Keyman Configuration user interface, or you can replace it entirely with your own interface.  In this example we will replace the user interface entirely.

Step 1: start Keyman Developer 7 and create a new project.  Add your keyboards and packages to the project.

Don't forget to compile your keyboards and packages.  This will generate normal .kmx and .kmp files.

Step 2: Create a branding file

Step 2.1: Create your default task bar icon for the branding file.

Step 2.2: Setup the product options.  This is where you specify the distribution model for the product

• Shareware: You define the length of the evaluation - there is no up-front licensing cost for Keyman, and you can start selling the product immediately through the Tavultesoft online store.  The online store automatically manages the licensing for Keyman and product activation.
• Freeware: You purchase a license to distribute Keyman with your product as freeware to all users - at no cost to them.  You cannot sell your product with this model but must make it freely available to all users.  No product activation is included in a freeware product.
• Global: You purchase a license to distribute Keyman with your product and you have full control over how you manage the product.  Product activation is not included in this model.

In our example, we will go with a 30 day evaluation Shareware product:

There is no need to change the controller file name in this example - we will use keyman.exe which is included in the Keyman Engine.  Next, we setup the application command lines - Keyman's COM API uses this when another application needs to activate your keyboard:

If you were using the standard Keyman Configuration user interface, you would use the following options instead:

You do not need to update any of the messages from Keyman Configuration if you are using your own user interface for the product.

Step 2.3: Design the notification icon popup menu:

Step 3: Back in the Project, add a new distribution file, and edit it:

Step 3.1: Select your custom product in the distribution licensing options page:

Step 3.2: Then, add all the files for your product to the package.  In this example, I have created a laounicode.exe .NET application that has a basic configuration dialog and splash screen.  I will cover the making of that in another blog entry - for now, you can download it from the Beta downloads site:

I have unticked the "Include Standard Keyman Files" checkbox as we are not using the Keyman Configuration user interface.

Please note: a bug in beta 1 means that you need to add the redistributable file insthelp.exe into the installer if you do not include Standard Keyman files.  insthelp.exe is available in the C:\Program Files\Tavultesoft\Keyman Developer\Redist folder.

Step 3.3: Fill in the install path and setup the readme and End User Licence Agreement files:

I don't think the optional program command line works in Beta 1...

Step 3.4: Fill in all the details shown in the Info page:

Step 3.5: Create Start Menu entries, as shown below (if you were using the Keyman Configuration UI, you would use kmshell.exe):

Step 4: Finally, save your distribution file and click the Create Product button.  You will need to be online to create the product.  All going well, a .msi file will be generated in the same path as your .kpp file:

Candle and light are part of the WiX toolkit that we use for creating Windows Installer MSI files.

During this process, you will probably be asked for your tavultesoft.com login details.  Keyman Developer requires access to online services from tavultesoft.com to encrypt your keyboard and setup product activation.

I will write about some other parts of the product creation process soon:

• Walkthrough the Tavultesoft Online product management website - product management, purchasing licenses, activating end user licenses and more.
• Dissecting the LaoUnicode.exe .NET example

Don't forget - if you encounter any bugs when following through this, create an issue!

Character Map

I was editing a keyboard in Keyman Developer 6 today (European Latin) and after using the Character Map in version 6 for a few minutes started to hanker for the version 7 Character Map that I have been using for the last few weeks while preparing for Beta 1.   In the end I had to run over to the test desk and use the version 7 Character Map.  I wanted to use the filter feature - especially the ability to use the * wildcard - I searched for "latin*ligature" so I could find all the Latin ligatures.  No way to do that in version 6.

In the image above, you can see some of the characters are in grey rather than black.  This indicates characters missing from the selected font and instead of skipping them, uses a backup font, in this case Code2000.  The character map also uses font linking to try and get a fallback character where one is missing - unfortunately, font linking does not appear to work in the supplementary planes.  James Kass's Code200x fonts provide great fallback coverage for the Character Map, with only a few scripts (mostly newer ones for the Unicode standard) missing, so we will be including them with Keyman Developer.

Of course, we still need to support showing all the characters in a given font - which is currently broken in beta 1.

The filtering currently supports a single phrase, with * and ? (or % and _) wildcards.  Any SQL fans will recognise the LIKE wildcard terms there.  You can also use [ and ] but we haven't tested that yet!  I'm not sure how valuable more complex filtering support is.

The automatic 'follow' feature in Keyman Developer 6 (where the character map shows the character under the cursor) doesn't really work all that well.  I think that we will make it follow only when a hotkey is pressed in Keyman Developer 7.

One other Good Thing: the version 7 Character Map uses Unicode 5.0.

The Beta Process

We released the first beta of Keyman and Keyman Developer 7 on Friday and were very pleased by the number of people who asked to join the beta in the first day.  They have already created 15 issue reports of high quality which is very encouraging (Join the beta).

After publishing the beta first thing on Friday, we spent the rest of the morning tearing down the wall between our two new offices - found it quite a good way of bleeding off the stress of putting out the release!  We have no more walls we can take down so I am not quite sure what we will do after publishing beta 2.

 

Yes, this is my first blog entry.  I tend to find it hard to write down this kind of thing and I am hoping that an informal outlet will be a good place to do that.  I do have a lot of things I want to write about regarding Keyman, so that will hopefully keep me going for a while.