Difference between revisions of "Strings and translation"

From PioneerWiki
Jump to: navigation, search
(Internationalization)
m (Lua Standard Library - String manipulation: missing paranthesis)
 
(16 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
Recommended reading: [https://pioneerwiki.com/wiki/Translations Pioneer translators wikipage]
 
Recommended reading: [https://pioneerwiki.com/wiki/Translations Pioneer translators wikipage]
  
It is important that scripters are familiar with the translation system. In the previous example '''hello_world.lua''' used strings directly like <code>print("Hello, World!")</code>. In order for Pioneer to be translated we instead use tokens and calls on the <code>'Lang'</code> module to substitute it for the string in the chosen language. The translated strings are stored together with their 'key' in json format, one language per file that are kept in the [https://github.com/pioneerspacesim/pioneer/tree/master/data/lang data/lang/] directory. If they are from the '''data/modules''' directory they have names starting with '''module-''' and ending with the name of the module written in one word, in small caps, and with the '.lua' ending omitted. The final translation from English to other languages is done on Pioneers project page at [https://www.transifex.com/pioneer/pioneer/ Transifex], a dedicated online translation tool. The translations are committed to the pioneer/master branch on GitHub by a bot and are not to edited manually apart from the '''en.json''' files.
+
It is important that scripters are familiar with the translation system. In the previous example '''hello_world.lua''' used strings directly like <code>print("Hello, World!")</code>. In order for Pioneer to be translated we instead use tokens and calls on the <code>'Lang'</code> module to substitute it for the string in the chosen language. The translated strings are stored together with their 'key' in json format, one language per file that are kept in the [https://github.com/pioneerspacesim/pioneer/tree/master/data/lang data/lang/] directory. If the script file in which they are used is from the '''data/modules''' directory they have names starting with '''module-''' and ending with the name of the module written in one word, in small caps, and with the '.lua' ending omitted. The final translation from English to other languages is done on Pioneer's project page at [https://www.transifex.com/pioneer/pioneer/ Transifex], a dedicated online translation tool. The translations are committed to the pioneer/master branch on GitHub by a bot and are not to be edited manually apart from the '''en.json''' files.
  
 
==Translating strings==
 
==Translating strings==
Line 17: Line 17:
 
And a corresponding file containing the translated string, and a description field shown to the translator, giving the context.  
 
And a corresponding file containing the translated string, and a description field shown to the translator, giving the context.  
  
'''data/lang/hello/en.json'''
+
'''data/lang/module-hello/en.json'''
 
  {
 
  {
  "HELLO": {
+
    "HELLO": {
    "description": "A classic message.",
+
        "description": "A classic message.",
    "message": "Hello World!"
+
        "message": "Hello World!"
  }
+
    }
 
  }
 
  }
  
 
In short, any script containing strings that needs translation needs to load the <code>'Lang'</code> module:
 
In short, any script containing strings that needs translation needs to load the <code>'Lang'</code> module:
 
  local Lang = require 'Lang'
 
  local Lang = require 'Lang'
Then declare a translator function, often just named <code>'l'</code> as in:
+
Then load the translation resource they want to use in their module, often just named <code>'l'</code> as in:
 
  local l = Lang.GetResource("module-hello")
 
  local l = Lang.GetResource("module-hello")
 
+
You may load more than one translation resource in your script but they will need to have unique names:
Note: Selecting any other language than a non-English, e.g. Esperanto, will trigger the game to crash, as it will try to replace the english string with a non-existing Esperanto file. This is only an issue when you're working on your code locally, once the script is merged into the master branch, Transifex will duplicate it to all supported languages, and copy it back into the main source repo.
+
local lh = Lang.GetResource("module-hello")
 +
local lg = Lang.GetResource("module-goodbye")
  
 
==Working with strings==
 
==Working with strings==
Line 38: Line 39:
 
[https://www.lua.org/pil/3.4.html 3.4 – Concatenation]<br>
 
[https://www.lua.org/pil/3.4.html 3.4 – Concatenation]<br>
 
[https://www.lua.org/pil/20.html 20 – The String Library]<br>
 
[https://www.lua.org/pil/20.html 20 – The String Library]<br>
And from the [https://www.lua.org/manual/5.0/ Lua 5.0 Reference Manual]<br>
+
And from the [https://www.lua.org/manual/5.2/ Lua 5.2 Reference Manual]<br>
[https://www.lua.org/manual/5.0/manual.html#5.3 5.3 – String Manipulation]<br>
+
[https://www.lua.org/manual/5.2/manual.html#6.4 6.4 – String Manipulation]<br>
  
 
===Concatenation===
 
===Concatenation===
Line 58: Line 59:
 
   
 
   
 
  local onShipDocked = function (ship)
 
  local onShipDocked = function (ship)
  if ship == Game.player then
+
    if ship == Game.player then
    Comms.Message(GREETING .. ' ' .. FBIAGENT .. '!')
+
        Comms.Message(GREETING .. ' ' .. FBIAGENT .. '!')
  end
+
    end
 
  end
 
  end
 
   
 
   
Line 69: Line 70:
 
===Lua Standard Library - String manipulation===
 
===Lua Standard Library - String manipulation===
  
I would start to search the '''data/''' register for ''''string.'''' to get a feeling for how it's used. Again on Linux i would use the 'grep' command. <code>grep -R "string\." data\</code>. You will see that the two most use cases is <code>string.format(</code> which is a part of the lua standard library, and <code>string.interp(</code> which is pulled into the script via the <code>'Lang'</code> module and is described in the next paragraph.
+
I would start to search the '''data/''' register for ''' 'string' '''. to get a feeling for how it's used. Again on Linux i would use the 'grep' command. <code>grep -R "string\." data/</code>. You will see that the two most frequently used functions is <code>string.format()</code>, which is a part of the lua standard library, and <code>string.interp()</code>, which is a Pioneer global function residing in '''data/libs/autoload.lua''', and is described in the next paragraph about [https://pioneerwiki.com/wiki/Strings_and_translation#String_variables String variables].
  
Some examples:
+
Some examples from the Lua standard library:
  Comms.Message(string.lower('wHaT cAsE sHoUlLd I usE?'))
+
  Comms.Message(string.lower('wHaT cAsE sHoULd I usE?'))
  
Maybe not the most easy function to fit into Pioneer. But it's there:
+
Maybe not the most easy function to fit into Pioneer, but it's there:
 
  Comms.Message('Today is opposite day!')
 
  Comms.Message('Today is opposite day!')
  Comms.Message(string.reverse('No, today is not opposite day!')
+
  Comms.Message(string.reverse('No, today is not opposite day!'))
  
 
A more complex example from '''[https://github.com/pioneerspacesim/pioneer/blob/452e9468bc32b844b379307f982c5869733bd85b/data/pigui/modules/hyperjump-planner.lua#L140 data/pigui/modules/hyperjump-planner.lua]'''<br>
 
A more complex example from '''[https://github.com/pioneerspacesim/pioneer/blob/452e9468bc32b844b379307f982c5869733bd85b/data/pigui/modules/hyperjump-planner.lua#L140 data/pigui/modules/hyperjump-planner.lua]'''<br>
Here <code>string.format("%.2f", distance)</code> is used to set the number of decimals in the jump to two. The rest is string variables that are not translated (names) and string concatenation. The translator is named ' '''lc''' '.<br>
+
Here <code>string.format("%.2f", distance)</code> is used to set the number of decimals in the jump to two. The rest is string variables that are not translated (names) and string concatenation. '''lc''' is used as the short name for the '''core''' translation resource.<br>
 
  textLine = jumpIndex ..": ".. jump_sys.name .. " (" .. string.format("%.2f", distance) .. lc.UNIT_LY .. " - " .. fuel .. lc.UNIT_TONNES..")"
 
  textLine = jumpIndex ..": ".. jump_sys.name .. " (" .. string.format("%.2f", distance) .. lc.UNIT_LY .. " - " .. fuel .. lc.UNIT_TONNES..")"
  
Here is what the final formatted text looks like (right side, two selected jump routes):
+
Here is what the final formatted text looks like (right side, two selected jump routes):<br>
 
[[File:hyperjumpplanner.png]]
 
[[File:hyperjumpplanner.png]]
  
Line 98: Line 99:
  
 
'''data/lang/module-stationrefuelling/en.json'''
 
'''data/lang/module-stationrefuelling/en.json'''
  "WELCOME_ABOARD_STATION_FEE_DEDUCTED": {
+
"WELCOME_TO_STATION_FEE_DEDUCTED": {
    "description": "Station welcome message, docking in orbit",
+
    "description": "Station welcome message, ground station landing",
    "message": "Welcome aboard {station}. Your docking fee of {fee} has been deducted."
+
    "message": "Welcome to {station}. Your landing fee of {fee} has been deducted."
  },
+
}
 +
 
 +
'''data/lang/module-stationrefuelling/it.json'''
 +
"WELCOME_TO_STATION_FEE_DEDUCTED": {
 +
    "description": "Station welcome message, ground station landing",
 +
    "message": "Benvenuti a {station}. Vi è stata dedotta una tassa di atterraggio di {fee}."
 +
}
 +
 
 +
==Mission flavours==
 +
 
 +
A script can define a mission and then place many instances of it onto many bulletin boards. A script that introduces many instances in the game should provide some variety; it would harm immersion if, for examle, all delivery missions were worded identically. The easiest way to achieve multiple versions, or "flavours", is by just making a number of different strings and then choosing between them with a simple random function. This is how the [https://github.com/pioneerspacesim/pioneer/blob/da538b07538962a5473cf02e1da0d9b615698772/data/lang/module-deliverpackage/en.json#L18:L48 deny] and [https://github.com/pioneerspacesim/pioneer/blob/da538b07538962a5473cf02e1da0d9b615698772/data/lang/module-deliverpackage/en.json#L282:L320 pirate taunt] messages are done in the DeliverPackage module.
 +
 
 +
'''data/modules/DeliverPackage/DeliverPackage.lua'''
 +
local num_deny = 8
 +
 +
        ...
 +
 +
if not qualified then
 +
    local introtext = l["DENY_"..Engine.rand:Integer(1,num_deny)-1]
 +
        form:SetMessage(introtext)
 +
    return
 +
end
 +
 
 +
'''data/lang/module-deliverpackage/en.json'''
 +
"DENY_0": {
 +
    "description": "",
 +
    "message": "I'm sorry, but I don't think I can trust you with this delivery."
 +
},
 +
"DENY_1": {
 +
    "description": "",
 +
    "message": "Come back when you have some qualifications."
 +
 +
        ...
 +
 +
"DENY_7": {
 +
    "description": "",
 +
    "message": "I'm not willing to risk this delivery on someone without any qualifications like you."
 +
You don't have to use a string variable that has been passed to the translation file. The pirate taunts, for example, have access to the clients name and the destination of the package but this information is only used in some of the strings.
 +
 +
"PIRATE_TAUNTS_7": {
 +
    "description": "",
 +
    "message": "That package isn't going to reach its destination today."
 +
},
 +
"PIRATE_TAUNTS_8": {
 +
    "description": "",
 +
    "message": "You're not getting to {location} today!"
 +
Another way to add variation is to have a table of strings that are variations of a theme, a flavour. In the example below, again from the DeliverPackage module, you have two of the lines from '''FLAVOUR_0''' AND '''FLAVOUR_1''' compared.
 +
"FLAVOUR_0_ADTEXT": {
 +
    "description": "",
 +
    "message": "GOING TO the {system} system? Money paid for delivery of a small package."
 +
},
 +
"FLAVOUR_0_FAILUREMSG": {
 +
    "description": "",
 +
    "message": "Unacceptable! You took forever over that delivery. I'm not willing to pay you."
 +
 +
        ...
 +
 +
"FLAVOUR_1_ADTEXT": {
 +
    "description": "",
 +
    "message": "WANTED. Delivery of a package to the {system} system."
 +
},
 +
"FLAVOUR_1_FAILUREMSG": {
 +
    "description": "",
 +
    "message": "I'm frustrated by the late delivery of my package, and I refuse to pay you."
 +
There are ten flavours in DeliverPackage of five strings each, each forming a short story line. They are static in that you only see lines from within the same flavour. You know what lines will follow if you accept the mission and after a short time of playing you know this already from seeing the ad if delivering things is your forte. It's hard to share strings in between the flavours as they would need to be much more similar. It's a trade off. You could build a very complex interaction with the customers but in the end it needs to be translatable.
  
'''data/lang/module-stationrefuelling/es.json'''
+
Generally, the more specific the text is, the stranger it will appear to see it multiple times in the game. For example, consider the following two examples:
  "WELCOME_ABOARD_STATION_FEE_DEDUCTED": {
+
- "Contract is to move X tonnes of Y to system Z"
    "description": "Station welcome message, docking in orbit",
+
- "My uncle's cat got sick, so I can't travel right now, so I need you to take my contract for me, to move X tonnes of Y to my home system Z"
    "message": "Bienvenido a bordo de {station}. Le ha sido deducida su cuota de atraque de {fee}."
+
The former might not need any flavours, as it's unpersonal, and very general. Seeing the second string multiple times does break immersion. Solution is to not show it often, either by making the mission rare / single occurance (no script to date does this) or have many flavours, making probability of the same twice low.
  },
 

Latest revision as of 17:33, 18 December 2022

Internationalization

Recommended reading: Pioneer translators wikipage

It is important that scripters are familiar with the translation system. In the previous example hello_world.lua used strings directly like print("Hello, World!"). In order for Pioneer to be translated we instead use tokens and calls on the 'Lang' module to substitute it for the string in the chosen language. The translated strings are stored together with their 'key' in json format, one language per file that are kept in the data/lang/ directory. If the script file in which they are used is from the data/modules directory they have names starting with module- and ending with the name of the module written in one word, in small caps, and with the '.lua' ending omitted. The final translation from English to other languages is done on Pioneer's project page at Transifex, a dedicated online translation tool. The translations are committed to the pioneer/master branch on GitHub by a bot and are not to be edited manually apart from the en.json files.

Translating strings

Below is a minimal example with one string added into the translation system. It takes two files, one in data/modules and one in data/lang/module-hello. The print statement isn't wrapped in a function so it will be executed when the hello.lua is being parsed on startup. You will have to scroll through the command-line history to find it interleaved with the other output.

data/modules/hello.lua.

local Lang = require 'Lang'
local l = Lang.GetResource("module-hello")

print(l.HELLO)

And a corresponding file containing the translated string, and a description field shown to the translator, giving the context.

data/lang/module-hello/en.json

{
    "HELLO": {
        "description": "A classic message.",
        "message": "Hello World!"
    }
}

In short, any script containing strings that needs translation needs to load the 'Lang' module:

local Lang = require 'Lang'

Then load the translation resource they want to use in their module, often just named 'l' as in:

local l = Lang.GetResource("module-hello")

You may load more than one translation resource in your script but they will need to have unique names:

local lh = Lang.GetResource("module-hello")
local lg = Lang.GetResource("module-goodbye")

Working with strings

Recommended chapters from Programming in Lua (first edition)
2.4 – Strings
3.4 – Concatenation
20 – The String Library
And from the Lua 5.2 Reference Manual
6.4 – String Manipulation

Concatenation

Instead of serving ready made sentences we can stitch them together with the Lua string concatenation operator ... The following codelines result in the same output.

print("Hello Clarice!")
print("Hello" .. " " .. "Clarice!")

This could be created by the more generic:

local Comms = require 'Comms'
local Event = require 'Event'
local Game = require 'Game'
local NameGen = require 'NameGen'

local GREETING = 'Hello'
local FBIAGENT = NameGen.Surname()

local onShipDocked = function (ship)
    if ship == Game.player then
        Comms.Message(GREETING .. ' ' .. FBIAGENT .. '!')
    end
end

Event.Register("onShipDocked", onShipDocked)

The .. operator is used all over the codebase. Try a search for its occurrence. On Linux I would do grep -R " \.\. " data/modules/ data/libs/ data/pigui/ and that will render a lot of output.

Lua Standard Library - String manipulation

I would start to search the data/ register for 'string' . to get a feeling for how it's used. Again on Linux i would use the 'grep' command. grep -R "string\." data/. You will see that the two most frequently used functions is string.format(), which is a part of the lua standard library, and string.interp(), which is a Pioneer global function residing in data/libs/autoload.lua, and is described in the next paragraph about String variables.

Some examples from the Lua standard library:

Comms.Message(string.lower('wHaT cAsE sHoULd I usE?'))

Maybe not the most easy function to fit into Pioneer, but it's there:

Comms.Message('Today is opposite day!')
Comms.Message(string.reverse('No, today is not opposite day!'))

A more complex example from data/pigui/modules/hyperjump-planner.lua
Here string.format("%.2f", distance) is used to set the number of decimals in the jump to two. The rest is string variables that are not translated (names) and string concatenation. lc is used as the short name for the core translation resource.

textLine = jumpIndex ..": ".. jump_sys.name .. " (" .. string.format("%.2f", distance) .. lc.UNIT_LY .. " - " .. fuel .. lc.UNIT_TONNES..")"

Here is what the final formatted text looks like (right side, two selected jump routes):
Hyperjumpplanner.png

String variables

When you want a string to present variable data such as numbers and names you use the interp() function. interp() accepts a table '{}' and the corresponding keys are then embedded in the translated strings enclosed by curly brackets '{}'.

Using 'module-stationrefuelling' we can do:

print(l.WELCOME_TO_STATION_FEE_DEDUCTED:interp({station = 'JFK', fee = 50}))

Example from the actual StationRefuelling module:

data/modules/StationRefuelling/

Comms.Message(l.WELCOME_TO_STATION_FEE_DEDUCTED:interp({station = station.label,fee = Format.Money(fee)}))

data/lang/module-stationrefuelling/en.json

"WELCOME_TO_STATION_FEE_DEDUCTED": {
    "description": "Station welcome message, ground station landing",
    "message": "Welcome to {station}. Your landing fee of {fee} has been deducted."
}

data/lang/module-stationrefuelling/it.json

"WELCOME_TO_STATION_FEE_DEDUCTED": {
    "description": "Station welcome message, ground station landing",
    "message": "Benvenuti a {station}. Vi è stata dedotta una tassa di atterraggio di {fee}."
}

Mission flavours

A script can define a mission and then place many instances of it onto many bulletin boards. A script that introduces many instances in the game should provide some variety; it would harm immersion if, for examle, all delivery missions were worded identically. The easiest way to achieve multiple versions, or "flavours", is by just making a number of different strings and then choosing between them with a simple random function. This is how the deny and pirate taunt messages are done in the DeliverPackage module.

data/modules/DeliverPackage/DeliverPackage.lua

local num_deny = 8

        ...

if not qualified then
    local introtext = l["DENY_"..Engine.rand:Integer(1,num_deny)-1]
        form:SetMessage(introtext)
    return
end

data/lang/module-deliverpackage/en.json

"DENY_0": {
    "description": "",
    "message": "I'm sorry, but I don't think I can trust you with this delivery."
},
"DENY_1": {
    "description": "",
    "message": "Come back when you have some qualifications."

        ...

"DENY_7": {
    "description": "",
    "message": "I'm not willing to risk this delivery on someone without any qualifications like you."

You don't have to use a string variable that has been passed to the translation file. The pirate taunts, for example, have access to the clients name and the destination of the package but this information is only used in some of the strings.

"PIRATE_TAUNTS_7": {
    "description": "",
    "message": "That package isn't going to reach its destination today."
},
"PIRATE_TAUNTS_8": {
    "description": "",
    "message": "You're not getting to {location} today!"

Another way to add variation is to have a table of strings that are variations of a theme, a flavour. In the example below, again from the DeliverPackage module, you have two of the lines from FLAVOUR_0 AND FLAVOUR_1 compared.

"FLAVOUR_0_ADTEXT": {
    "description": "",
    "message": "GOING TO the {system} system? Money paid for delivery of a small package."
},
"FLAVOUR_0_FAILUREMSG": {
    "description": "",
    "message": "Unacceptable! You took forever over that delivery. I'm not willing to pay you."

        ...

"FLAVOUR_1_ADTEXT": {
    "description": "",
    "message": "WANTED. Delivery of a package to the {system} system."
},
"FLAVOUR_1_FAILUREMSG": {
    "description": "",
    "message": "I'm frustrated by the late delivery of my package, and I refuse to pay you."

There are ten flavours in DeliverPackage of five strings each, each forming a short story line. They are static in that you only see lines from within the same flavour. You know what lines will follow if you accept the mission and after a short time of playing you know this already from seeing the ad if delivering things is your forte. It's hard to share strings in between the flavours as they would need to be much more similar. It's a trade off. You could build a very complex interaction with the customers but in the end it needs to be translatable.

Generally, the more specific the text is, the stranger it will appear to see it multiple times in the game. For example, consider the following two examples:

- "Contract is to move X tonnes of Y to system Z"
- "My uncle's cat got sick, so I can't travel right now, so I need you to take my contract for me, to move X tonnes of Y to my home system Z"

The former might not need any flavours, as it's unpersonal, and very general. Seeing the second string multiple times does break immersion. Solution is to not show it often, either by making the mission rare / single occurance (no script to date does this) or have many flavours, making probability of the same twice low.