C-LARA

We were away for a bit more than a week while Cathy was playing a bridge tournament, so I haven’t been working on C-LARA as much as usual during this period. I have however been experimenting with using GPT-4 in a different context, which has turned up some interesting ideas with parallelism and few-shot examples that should be relevant to our project.

Coherent image sets

A new C-LARA feature I have added is the possibility of creating coherent image sets from translations rather than from the original text. This is useful for languages that aren’t supported by the AI.

The feature is easy to use: you specify when you create the project, or in the project menu, that you are using translations for the coherent images, and you fill in segment translations before you create the coherent images.

A side experiment: using Chain of Thought to play Tic-Tac-Toe

This is an idea that occurred to me a couple of weeks ago, and while Cathy was playing bridge I couldn’t resist the temptation to investigate it. Chain of Thought reasoning is becoming increasingly important when using LLMs, but, as for example pointed out in the Aschenbrenner essay, there is a shortage of training material. It would be extremely useful to be able to create such material automatically.

Following in the footsteps of AlphaZero, I thought we could try using the game of Tic-Tac-Toe. GPT-4 is notoriously bad at it, so the problem isn’t trivial. The idea would be to have GPT-4 play Tic-Tac-Toe using a CoT method, and recycle the successful CoT protocols as few-shot examples. The hope is that the CoT player will gradually improve.

So far, we do not see any clear improvement, but there are some interesting initial results, and a couple of things have turned up which look like they will be very useful in C-LARA:

Parallelism

It took a long time to run the experiments, and I asked ChatGPT-4o if we could parallelise things, playing multiple games at the same time. We had discussed this before, and Chat had said it was easy, but I wasn’t sure I believed it. Well: it turns out it is easy! We were able to parallelise the Tic-Tac-Toe code in less than a day, with Chat doing nearly all the work, and it speeds things up a great deal.

C-LARA is probably a bit more complicated, but the same principles will apply, and now that I’ve played with a live example I’m sure it will work. We can use this method for annotation: instead of submitting the annotation tasks as a sequence, we can run them all at the same time. In the same way, we can submit multiple image generation requests simultaneously, as long as the later requests don’t depend on the earlier ones. For “flat” tasks like generation of picture dictionaries, this should work well.

Few-shot examples

One thing that soon became clear with Tic-Tac-Toe is that it helps to match your few-shot examples to the position: you should pick examples that are as similar as possible.

Again, it’s harder with linguistic annotation, but the same idea is likely to help. Particularly in glossing, a task we still find challenging, it’s easy to imagine that it will be useful to give the AI few-shot examples which contain words and phrases similar to those in the segment we’re trying to gloss. The most obvious way to define “similar” would be in terms of n-grams of POS tags.

I suggest we start looking at both of these ideas in the near future, particularly with reference to producing Ukrainian-glossed texts.

Next Zoom call

The next call will be at:

Thu Jul 18 2024, 18:00 Adelaide (= 08.30 Iceland = 09.30 Ireland/Faroe Islands = 10.30 Europe = 11.30 Israel = 12.00 Iran = 16.30 China = 18:30 Melbourne = 19.30 New Caledonia)