[Solved] Getting PlatformIO working for Pokitto

jonne · July 24, 2018, 12:09pm

Success!

We have blinky from platformio.

I will still have to adapt a workaround that is automatic for other users.

2 bugfixes were needed. Both can be attributed to NXP not taking care of testing their compatibility with both mbed OS 2 and mbed OS 5. In fact, i suspect they have not tested their older boards on mbed OS2 after the migration to OS 5 at all.

I will go into detail as soon as i have time.

jonne · July 24, 2018, 9:43pm

#Help wanted

Do we have Python coders in the house? Task is quite simple: we need a so-called PRE build script (Python) that will download and replace the default linker script and startup_lpc11u68.c. This script runs before the build process of Platformio begins. If we can do this, the PlatformIO thing is almost done.

The other alternative is writing a new framework and that will be a lot more work. This idea of a pre script was suggested by PIO support themselves

Pharap · July 24, 2018, 10:43pm

I think @Hanski knows Python.
He was the one who got micro Python working if I remember rightly.

If not I could have a crack at it, but I’d need to know what the APIs are like.

Hanski · July 25, 2018, 5:32am

I have not used the Python http api, if that is what is needed. I am also this week on holiday in Spain.

Pharap · July 25, 2018, 6:56am

I could probably give it a go. I read a book about python many years ago (but never bothered to make anything more than hello world).

I’ve had this tab open for a long time waiting for the moment it would be needed:
http://docs.platformio.org/en/latest/projectconf/advanced_scripting.html

Is this a post-build action, a pre-build action or something else?
Where does the file sit relative to the build directory?
What url would it be fetching from?

Also, any idea what version of Python we’re dealing with?
It’s one of those languages where the versions can be drastically different

So far my best guess is something like this (untested):

import urllib.request

Import("env")

def downloadCFile
	url = 'https://talk.pokitto.com/uploads/default/original/2X/a/a603fd63612774c804183eca2103245922662f8a.gif'
	urllib.request.urlretrieve(url, "$BUILD_DIR/startup_lpc11u68.c")

# Download script before building
env.AddPreAction("buildprog", downloadCFile)

jonne · July 25, 2018, 9:58pm

#LPC11U68 target is working automatically now

This is a system that automatically gets startup_LPC11U68.cpp and the LPC11U68.ld linker file from our PokittoLib github repository and patches the broken versions in the PlatformIO mbed framework. it also checks if any substantial changes have occurred by comparing existing files with those found in the github repo.

Note: this can not make Pokitto programs yet. It makes fully working mbed binaries now. PokittoLib will be added to the build process soon. This is a work in progress!

Note 2: upload protocol should be changed to a logical disk drive (i.e. not JLink) unfortunately I don’t have time to do it now. I know it can be done, google it. I got Pokitto drive as upload protocol to work on a previous try at PlatformIO compilation.

Steps:
Install VSCODE or Atom (Atom not tested by me)
Install PlatformIO extension
Make new PlatformIO project, select LPCXpresso68 as board

edit your platformio.ini

; PlatformIO Project Configuration File
;
;   Build options: build flags, source filter
;   Upload options: custom upload port, speed and extra flags
;   Library options: dependencies, extra library storages
;   Advanced options: extra scripting
;
; Please visit documentation for the other options and examples
; http://docs.platformio.org/page/projectconf.html

[env:lpc11u68]
platform = nxplpc
board = lpc11u68
framework = mbed
build_flags =
  -DPOKITTO_PIO_BUILD
extra_scripts = pre:pokitto_pre.py
upload_protocol = jlink
debug_tool = custom
debug_server =
    usr/bin/JLinkGDBServer
    -singlerun
    -if
    SWD
    -select
    USB
    -port
    2331
    -device
    LPC11U68

add this script pokitto_pre.py as a file to the same directory as platformio.ini

import requests
import os
import filecmp

#the following is needed, because at this stage PLATFORMIO_HOME_DIR is undefined
from os.path import expanduser
home = expanduser("~")

if os.name == 'nt': # Windows
    basePath = home + '\.platformio'
else:
    basePath = home + '/.platformio'

patchPath = basePath + '/packages/framework-mbed/targets/TARGET_NXP/TARGET_LPC11U6X/device/TOOLCHAIN_GCC_ARM/TARGET_LPC11U68/'

#get the latest startup files from github
print("Comparing startup_LPC11U68.cpp to PokittoLib repository...")
url = 'https://raw.githubusercontent.com/pokitto/PokittoLib/master/Pokitto/mbed-pokitto/targets/cmsis/TARGET_NXP/TARGET_LPC11U6X/TOOLCHAIN_GCC_ARM/TARGET_LPC11U68/startup_LPC11U68.cpp'
r = requests.get(url, allow_redirects=True, headers={'Cache-Control': 'no-cache'})
open(patchPath + 'startup_LPC11U68.tmp', 'wb').write(r.content)

#check if startup_LPC11U68.bak exists
if not (os.path.exists(patchPath + 'startup_LPC11U68.bak')):
    #first run, so create a backup
    print('Creating backup of original startup_LPC11U68.cpp')
    os.rename(patchPath + 'startup_LPC11U68.cpp', patchPath + 'startup_LPC11U68.bak')
    
#compare new .tmp file(s) to existing files
if (os.path.exists(patchPath + 'startup_LPC11U68.cpp')):
    if not filecmp.cmp(patchPath + 'startup_LPC11U68.tmp', patchPath + 'startup_LPC11U68.cpp'):
        #they are different, so update
        print('New version found. Saving it as startup_LPC11U68.cpp')
        open(patchPath + 'startup_LPC11U68.cpp', 'wb').write(r.content)
else:
    #missing completely, so save
    print('Saving startup_LPC11U68.cpp')
    open(patchPath + 'startup_LPC11U68.cpp', 'wb').write(r.content)

#delete temporary file(s)
os.remove(patchPath+'startup_LPC11U68.tmp')

#get the latest linker file from github
print("Comparing linker file LPC11U68.ld to PokittoLib repository...")
url = 'https://raw.githubusercontent.com/pokitto/PokittoLib/master/Pokitto/mbed-pokitto/targets/cmsis/TARGET_NXP/TARGET_LPC11U6X/TOOLCHAIN_GCC_ARM/TARGET_LPC11U68/LPC11U68.ld'
r = requests.get(url, allow_redirects=True, headers={'Cache-Control': 'no-cache'})
open(patchPath + 'LPC11U68.tmp', 'wb').write(r.content)

#check if LPC11U68.bak exists
if not (os.path.exists(patchPath + 'LPC11U68.bak')):
    #first run, so create a backup
    if (os.path.exists(patchPath + 'LPC11U68.ld')):
        print('Creating backup of original LPC11U68.ld')
        os.rename(patchPath + 'LPC11U68.ld', patchPath + 'LPC11U68.bak')
    
#compare new .tmp file(s) to existing files
if (os.path.exists(patchPath + 'LPC11U68.ld')):
    if not filecmp.cmp(patchPath + 'LPC11U68.tmp', patchPath + 'LPC11U68.ld'):
        #they are different, so update
        print('New version found. Saving it as LPC11U68.ld')
        open(patchPath + 'LPC11U68.ld', 'wb').write(r.content)
else:
    #missing completely, so save
    print('Saving LPC11U68.ld')
    open(patchPath + 'LPC11U68.ld', 'wb').write(r.content)   

#delete temporary file(s)
os.remove(patchPath+'LPC11U68.tmp')

Expected output in VSCODE terminal:

> Executing task: platformio run <

Processing lpc11u68 (platform: nxplpc; board: lpc11u68; framework: mbed)
----------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
Comparing startup_LPC11U68.cpp to PokittoLib repository...
Comparing linker file LPC11U68.ld to PokittoLib repository...
Saving LPC11U68.ld
PLATFORM: NXP LPC > LPCXpresso11U68
SYSTEM: LPC11U68 50MHz 36KB RAM (256KB Flash)
DEBUG: CURRENT(custom) ON-BOARD(cmsis-dap) EXTERNAL(blackmagic, jlink)
Library Dependency Finder -> http://bit.ly/configure-pio-ldf
LDF MODES: FINDER(chain) COMPATIBILITY(soft)
Collected 15 compatible libraries
Scanning dependencies...
No dependencies
Generating LD script .pioenvs/lpc11u68/LPC11U68.ld.link_script.ld
Linking .pioenvs/lpc11u68/firmware.elf
Checking size .pioenvs/lpc11u68/firmware.elf
Memory Usage -> http://bit.ly/pio-memory-usage
DATA:    [          ]   0.4% (used 140 bytes from 36864 bytes)
PROGRAM: [          ]   1.4% (used 3560 bytes from 262144 bytes)
Building .pioenvs/lpc11u68/firmware.bin
================================ [SUCCESS] Took 7.62 seconds ================================

Terminal will be reused by tasks, press any key to close it.

Pharap · July 25, 2018, 10:28pm

This is excellent work.
I’ll get round to testing it at some point.

There’s no rush to keep working on this, we know you’re busy with other things.

jonne · July 25, 2018, 10:40pm

Thanks!

Two observations:

I love the vscode + platformio user experience. When it works, its miles better than for example Arduino IDE. Way better. I should say even for a beginner. The UI is very responsive, visual formatting is unobtrusive, function/symbol hints appear at the right time. When we get this PIO thing to work 100% its going to be a strong contender for the best toolchain/IDE, no question about it
Now that I’ve understood how the build system works, especially the scripting, I have realized you can get this thing to do anything you want. The customizability is on a totally different level to what I’ve used before

Pharap · July 25, 2018, 11:03pm

This is precisely why I refused to let us give up hope on PlatformIO.

I could see the potential that such a setup would have,
and I could see that having PIO+VSCode makes the Pokitto more attractive because VSCode is quite popular at the moment.
(Programmers can be very picky about their tools.)

When I was trying on my own I just didn’t know enough about the build process to make much progress.

(In light of the discovery of that broken function I think it’s possible that I did have it working at one point but was encountering that function problem.)

Now that we’re on our way to having both Code::Blocks and VSCode (and possibly Atom) available,
I’m hoping that we’ll end up with some more regular users.

jonne · July 25, 2018, 11:09pm

You probably did. But the bug would have been impossible to track down. Like I said, I used Segger Ozone, a fully standalone hw debugging suite before I got a clue as to what was wrong

FManga · July 26, 2018, 2:05pm

Does pokitto_pre.py run before every build? Wouldn’t it be better if it only tried to download an update if the bak wasn’t present? If no internet connection is available, won’t that break even if you had downloaded the files previously?

jonne · July 26, 2018, 2:35pm

We have Hello Platformio

Its 100% functional. Edit: only tested on Ubuntu Linux so far, upload still JLink

jonne · July 27, 2018, 5:41am

#update

I have packaged and registered Pokitto library to PlatformIO library registy

It seems to take >24hours for PIO library registry entries to be approved

Waiting for the lib to appear

After that, getting started on PIO will be a piece of cake (no need to learn git etc)

adekto · July 27, 2018, 2:59pm

this library will work for usb upload / sd card bin?

jonne · July 27, 2018, 3:11pm

yes it will

adekto · July 27, 2018, 3:13pm

so this is going to be the new way for beginners to program for pokitto right?

(considering doing the tutorials)

jonne · July 27, 2018, 3:20pm

Yes but the approval for the library into the Platformio reg is taking ages

Why I would like it to be registered is it makes importing the lib a lot easier

adekto · July 27, 2018, 3:26pm

yea thats understandable

so just to be clear we install atom/visual studio code, then Platformio and in there is a pokitto start project?

jonne · July 27, 2018, 3:40pm

Yes, the idea is that you install PlatformIO, import the lib through the lib manager and you’re done

Pharap · July 27, 2018, 5:07pm

I don’t know about Atom, but in VSCode is basically a fancy text editor and anything more advanced than editing plaintext and managing keybinds is a downloadable extension,
including the C++ language and PlatformIO.