Android FTDI support not working with fresh app build

Ok, there is something very strange going on.
Here is what I've done.
I started over. I completely removed the subsurface directory and re-cloned the subsurface repo.
Followed the instructions in the INSTALL file and built an android .apk image from the head of the tree.
It was slightly larger than the previous .apk image I had previously built that wasn't able to open the ftdi port.
This newly built app did not show up when the ftdi cable was plugged in (neither did the first one)
When the app was started manually, the app would come up and the immediately crash.
I decided to start over again clean and checkout v4.8.1 and see if I could build that.
The .apk built from the v4.8.1 code was installed, the app would show up when the cable was plugged in.
This newly built v4.8.1 build could talk to the FTDI cable just like the 4.8.1 app downloaded from the playstore.

Now for the REALLY strange stuff!
The timing that is wrong in the app installed from the playstore is now correct with the app I just built.
With the 4.8.1 app I just built, downloads work on both the Galaxy S4 and the S7.

I have a logic analyzer hooked to aÂ h/w & s/w simulator test jig that simulates the Pelagic data cable and an Atmos AI
to be able to monitor data and signals on the FTDI chip to see what the host is doing.
There is a critical delay that must happen between RTS dropping and the first command sent to the PIC inside the Pelagic data cable.
The delay should be 100ms (that is what libdivecomputer code is asking for.)

With the 4.8.1 apk from the play store, this delay is
2.5ms on the Galaxy S4
600us on the Galaxy S7
Both are way too short and that is why downloads don't work. The PIC inside the Pelagic datacable misses the command because it isn't listening yet.

With the 4.8.1 code I checkout from the repository and built on my machine, the delays are:

103ms on the Galaxy S4
100ms on the Galaxy S7
These are the proper delays and downloads work as they should with this build.

I can post logic analyzer timing diagrams if people are interested in that level of detail.

There is something very odd going on.
I'm at a total loss as to what is happening and why my build has working delays whereas
the code from the playstore appears to not have working delays.

--- bill

The Ubuntu package won't help, you need to build it for Android. I thought the scripts did that, but I'm not at my computer and trying to figure this out on the phone is now than I am ready for...
Â Â Â Â I figured as much.
Â Â Â Â I hadn't installed libusb-1.0.0-dev
Â Â Â Â so I went back an installed all the Ubuntu packages listed in the INSTALL file, including libusb-1.0.0-dev
Â Â Â Â and re-ran the build script. But no difference.
Â Â Â Â I must still be missing something, probably really simple.
Â Â Â Â Â Â Â Â Â Â Â Â Â The android app downloaded from the Playstore behaves differently than the one I just built from current sources.
Â Â Â Â Â Â Â Â Â Â Â Â Â I'm currently testing on a Samsung GS4 running kitkat 4.4.4
Â Â Â Â Â Â Â Â Â Â Â Â Â With the one from the play store: version 2.1.0(4.8.0)
Â Â Â Â Â Â Â Â Â Â Â Â Â - When the cable is plugged in OS asks to run subsurface.
Â Â Â Â Â Â Â Â Â Â Â Â Â NOTE: it will not do this if running the app in no cloud mode.
Â Â Â Â Â Â Â Â Â Â Â Â Â and it also won't talk over the ftdi cable either.
Â Â Â Â Â Â Â Â Â Â Â Â Â Once in cloud mode,
Â Â Â Â Â Â Â Â Â Â Â Â Â - It can open the port and push bytes out the data cable
Â Â Â Â Â Â Â Â Â Â Â Â Â However, the timing between the bytes & messages is wrong (which is what I'm trying to fix)
Â Â Â Â Â Â Â Â Â Â Â Â Â With the one I just built: version 2.1.2(4.8.1.413)
Â Â Â Â Â Â Â Â Â Â Â Â Â It appears to not be able to open the serial connection.
Â Â Â Â Â Â Â Â Â Â Â Â Â The libdivecomputer log shows: Subsurface: v4.8.1-413-g76c4fb397512, built with libdivecomputer v0.7.0-devel-Subsurface-NG (8f2ac8f61e8768d4774983de1f367f73c8c34ea2)
Â Â Â Â Â Â Â Â Â Â Â Â Â INFO: Open: name=ftdi
Â Â Â Â Â Â Â Â Â Â Â Â Â ERROR: No such file or directory (2) [in /home/bill/Documents/devel/Subsurface-devel/subsurface/libdivecomputer/src/serial_posix.c:295 (dc_serial_open)] Which seems to indicate it is trying open the device "ftdi" vs checking for the magic

"ftdi" name and calling the ftdi serial code instead of serial_posix code.

Â Â Â Â Â Â Â Â Â Â Â Â Â Have I not built the app properly, or is the code broken right now and I should go back to the tagged code from the last release?
Â Â Â Â Â Â Â Â Â You are building the app without libftdi
Â Â Â Â Â Â Â Â Â /D

Anton Lundin

2018-09-14 13:56:55 UTC

Post by Bill Perry
The android app downloaded from the Playstore behaves differently than the one I just built from current sources.

How did you build the mobile app for android?

Post by Bill Perry
I'm currently testing on a Samsung GS4 running kitkat 4.4.4
With the one from the play store: version 2.1.0(4.8.0)
- When the cable is plugged in OS asks to run subsurface.
NOTE: it will not do this if running the app in no cloud mode.
and it also won't talk over the ftdi cable either.
Once in cloud mode,

Nope. This has nothing to do with each other.

Post by Bill Perry
- It can open the port and push bytes out the data cable
However, the timing between the bytes & messages is wrong (which is what I'm trying to fix)
With the one I just built: version 2.1.2(4.8.1.413)
It appears to not be able to open the serial connection.
====================
Subsurface: v4.8.1-413-g76c4fb397512, built with libdivecomputer v0.7.0-devel-Subsurface-NG (8f2ac8f61e8768d4774983de1f367f73c8c34ea2)
INFO: Open: name=ftdi
ERROR: No such file or directory (2) [in /home/bill/Documents/devel/Subsurface-devel/subsurface/libdivecomputer/src/serial_posix.c:295 (dc_serial_open)]
====================
Which seems to indicate it is trying open the device "ftdi" vs checking for the magic "ftdi" name and calling the ftdi serial code instead of serial_posix code.
Have I not built the app properly, or is the code broken right now and I should go back to the tagged code from the last release?

You haven't built the app properly.

Post by Bill Perry
Here is the subsurface log
====================
"0.012: Successfully opened logfile /storage/emulated/0/subsurface.log at Thu Sep 13 19:49:06 2018"
"0.013: Starting Subsurface-mobile:2.1.2(4.8.1.413):Android KitKat (4.4):arm:en-US"
"0.013: built with libdivecomputer v0.7.0-devel-Subsurface-NG (8f2ac8f61e8768d4774983de1f367f73c8c34ea2)"
"0.014: built with Qt Version 5.11.1, runtime from Qt Version 5.11.1"
"0.014: built with libgit2 0.26.0"
"localDevice bperrybap-SGH-M919 is valid, starting discovery"
paired BT classic device type 1 with address "00:00:00:00:00:01"
paired BT classic device type 1 with address "B8:27:EB:AA:44:2B"
paired BT classic device type 1 with address "00:BA:55:56:45:2E"
Found new device: "obd-2" "00:00:00:00:00:01"
Not recognized as dive computer
Found new device: "GreenIceZero" "B8:27:EB:AA:44:2B"
Not recognized as dive computer
Found new device: "AB SHUTTER 3" "00:BA:55:56:45:2E"
Not recognized as dive computer
Paired = "obd-2" "00:00:00:00:00:01"
Paired = "GreenIceZero" "B8:27:EB:AA:44:2B"
Paired = "AB SHUTTER 3" "00:BA:55:56:45:2E"
"Created position source android"
"0.095: Created position source android"
"Set GPS service update interval to 300 s"
"0.096: Set GPS service update interval to 300 s"
"0.096: location service is available"
"1.702: Synchronising data file"
"1.707: Load dives from local cache"
"1.731: Successfully opened dive data"
"1.737: 26 dives loaded from cache"
"1.740: have cloud credentials, but user asked not to connect to network"
"Set GPS service update interval to 300 s"
"1.741: Set GPS service update interval to 300 s"
checkPendingIntents
Using the following font: Roboto
qqwindow devicePixelRatio 3 3
"Aeris: 500 AI (SERIAL), A300 (SERIAL), A300 AI (SERIAL), A300CS (SERIAL), Atmos 2 (SERIAL), Atmos AI (SERIAL), Atmos AI 2 (SERIAL), Compumask (SERIAL), Elite (SERIAL), Elite T3 (SERIAL), Epic (SERIAL), F10 (SERIAL), F11 (SERIAL), Manta (SERIAL), XR-1 NX
(SERIAL), XR-2 (SERIAL)"
"Aqualung: i200 (SERIAL), i300 (SERIAL), i450T (SERIAL), i550 (SERIAL), i750TC (SERIAL, BT)"
"Atomic Aquatics: Cobalt (USB), Cobalt 2 (USB)"
"Beuchat: Mundial 2 (SERIAL), Mundial 3 (SERIAL), Voyager 2G (SERIAL)"
"Cochran: Commander I (SERIAL), Commander II (SERIAL), Commander TM (SERIAL), EMC-14 (SERIAL), EMC-16 (SERIAL), EMC-20H (SERIAL)"
"Cressi: Drake (SERIAL), Giotto (SERIAL), Leonardo (SERIAL), Newton (SERIAL)"
"Garmin: Descent Mk1 (USBSTORAGE)"
"Genesis: React Pro (SERIAL), React Pro White (SERIAL)"
"Heinrichs Weikamp: Frog (SERIAL, BT), OSTC (SERIAL), OSTC 2 (SERIAL, BT, BLE), OSTC 2 TR (SERIAL, BT, BLE), OSTC 2C (SERIAL), OSTC 2N (SERIAL), OSTC 3 (SERIAL), OSTC 4 (SERIAL, BT, BLE), OSTC Mk2 (SERIAL), OSTC Plus (SERIAL, BT, BLE), OSTC Sport
(SERIAL, BT, BLE), OSTC cR (SERIAL)"
"Hollis: DG02 (SERIAL), DG03 (SERIAL), TX1 (SERIAL)"
"Mares: Puck Pro (SERIAL, BLE), Quad (SERIAL, BLE), Quad Air (SERIAL, BLE), Smart (SERIAL, BLE), Smart Air (SERIAL, BLE)"
"Oceanic: Atom 1.0 (SERIAL), Atom 2.0 (SERIAL), Atom 3.0 (SERIAL), Atom 3.1 (SERIAL), Datamask (SERIAL), F10 (SERIAL), F11 (SERIAL), Geo (SERIAL), Geo 2.0 (SERIAL), OC1 (SERIAL), OCS (SERIAL), OCi (SERIAL), Pro Plus 2 (SERIAL), Pro Plus 2.1 (SERIAL), Pro
Plus 3 (SERIAL), VT 4.1 (SERIAL), VT Pro (SERIAL), VT3 (SERIAL), VT4 (SERIAL), VTX (SERIAL), Veo 1.0 (SERIAL), Veo 180 (SERIAL), Veo 2.0 (SERIAL), Veo 200 (SERIAL), Veo 250 (SERIAL), Veo 3.0 (SERIAL), Versa Pro (SERIAL)"
"Scubapro: Aladin Sport Matrix (BLE), Aladin Square (USBHID), G2 (USBHID, BLE), G2 Console (USBHID, BLE)"
"Seemann: XP5 (SERIAL)"
"Shearwater: Nerd (SERIAL, BT), Nerd 2 (BLE), Perdix (SERIAL, BT, BLE), Perdix AI (BLE), Petrel (SERIAL, BT), Petrel 2 (SERIAL, BT, BLE), Predator (SERIAL, BT), Teric (BLE)"
"Sherwood: Amphos (SERIAL), Amphos Air (SERIAL), Insight (SERIAL), Insight 2 (SERIAL), Vision (SERIAL), Wisdom (SERIAL), Wisdom 2 (SERIAL), Wisdom 3 (SERIAL)"
"Subgear: XP-Air (SERIAL)"
"Suunto: Cobra (SERIAL), Cobra 2 (SERIAL), Cobra 3 (SERIAL), D3 (SERIAL), D4 (SERIAL), D4f (SERIAL), D4i (SERIAL), D6 (SERIAL), D6i (SERIAL), D9 (SERIAL), D9tx (SERIAL), DX (SERIAL), EON Core (USBHID, BLE), EON Steel (USBHID, BLE), Eon (SERIAL), Gekko
(SERIAL), HelO2 (SERIAL), Mosquito (SERIAL), Solution (SERIAL), Solution Alpha (SERIAL), Solution Nitrox (SERIAL), Spyder (SERIAL), Stinger (SERIAL), Vyper (SERIAL), Vyper 2 (SERIAL), Vyper Air (SERIAL), Vyper Novo (SERIAL), Vytec (SERIAL), Zoop
(SERIAL), Zoop Novo (SERIAL)"
"Tecdiving: DiveComputer.eu (SERIAL, BT)"
"Tusa: Element II (IQ-750) (SERIAL), Zen (IQ-900) (SERIAL), Zen Air (IQ-950) (SERIAL)"
"Uwatec: Aladin Air Twin (SERIAL), Aladin Air Z (SERIAL), Aladin Air Z Nitrox (SERIAL), Aladin Air Z O2 (SERIAL), Aladin Pro (SERIAL), Aladin Pro Ultra (SERIAL), Aladin Sport Plus (SERIAL), Memomouse (SERIAL)"
qqwindow screen has ldpi/pdpi 72 146.967
"4.798: AppState changed to active with no save ongoing and no unsaved changes"
"15.411: AppState changed to inactive with no save ongoing and no unsaved changes"
"20.321: AppState changed to active with no save ongoing and no unsaved changes"
"23.558: DCDownloadThread started for Aeris Atmos AI on FTDI"
Starting download from ftdi
"23.579: Looking at device with VID/PID 1478/36940"
"23.579: Looking at device with VID/PID 1027/24577"
"23.580: usbManager tells us we don't have permission to access this device"

This is the money shot: You didn't give Subsurface mobile access to this
usb device, thus android denies us access to it.

Post by Bill Perry
Finishing download thread: "Unable to open ftdi Aeris (Atmos AI)"
"23.591: Unsupported operation"
no new dives downloaded
"23.592: DCDownloadThread finished"
The item Settings_QMLTYPE_30(0x7ca5f068, "Settings") is already in the PageRow

//Anton

--
Anton Lundin +46702-161604

Bill Perry

2018-09-14 15:30:55 UTC

Post by Bill Perry
The android app downloaded from the Playstore behaves differently than the one I just built from current sources.

How did you build the mobile app for android?

note: did you see the latest strangeness: http://lists.subsurface-divelog.org/pipermail/subsurface/2018-September/033025.html
Where libdivecomputer command/message delays don't work on the 4.8.1 app from the playstore but do work with the 4.8.1 that I built.
This was verified with a logic analyzer with the apps running on a Galaxy S4 with Android 4.4.4 and a Galaxy S7 running Android 8.0.0

I followed the instructions from the INSTALL file, except the only package I added to a newly installed Mint 18 OS install was cmake.
I'm building on freshly installed fully up to date 64bit Linux Mint MATE 18.3 in a virtual box vm.
I forked the subsurface repository then cloned from my github repo.
I did the submodule update to get the proper libdivecomputer
I then ran bash subsurface/packaging/android/android-build-wrapper.sh to build the .apk
I never built the code for the desktop.

After I sent the first post about this, I went back and started over with clean clone of the repo and
I installed all the recommended Ubuntu 18.04 packages (even though Mint 18 is based on Ubuntu 16.04)
and rebuilt the app after a clean/fresh clone.
That build would come up but immediately crash on both Android 4.4.4 and Android 8.0.0
I then started over again with a clean/fresh clone and checked out v4.8.1 and then
followed the same steps to build the apk
Nothing else was changed on the machine other then I started with a clean clone and checked out v4.8.1 vs using the head of Master.
This apk worked as expected including having the proper command/message delays
whereas the current app from the app store (4.8.1) does not have the proper delays.

I'm at a loss to explain the delay issue, but I can see it on a logic analyzer.

Nope. This has nothing to do with each other.

I hear you and agree it shouldn't be related but I saw this more than once.
I wasn't focusing on this at the time so I just went ahead and connected to the cloud account so I
could get the serial port to spit out some data to look at the timing.
I'll go back and try to replicate and verify this and the details on if/how to replicate it
assuming I can replicate it.

Also, there is no way to download using the FTDI cable if Bluetooth is not enabled.
If bluetooth is not enabled, it won't let you download at all.

You haven't built the app properly.

I figured as much, but I didn't really know why.
I can only assume (perhaps incorrectly) that the subsurface/packaging/android/android-build-wrapper.sh script
looks for the existence of libusb being installed and if it doesn't see it installed doesn't include support for it in the mobile build.
The first build I did, I didn't have libusb-1.0-0-dev package installed on the machine.

The big clue for me is that dc_serial_open() is being called.
This only gets called for /dev type devices.
It should not be called for the fake/magic "ftdi" device filename that is used when selecting the "FTDI" device
from the download UI.

Post by Bill Perry
"23.558: DCDownloadThread started for Aeris Atmos AI on FTDI"
Starting download from ftdi
"23.579: Looking at device with VID/PID 1478/36940"
"23.579: Looking at device with VID/PID 1027/24577"
"23.580: usbManager tells us we don't have permission to access this device"

This is the money shot: You didn't give Subsurface mobile access to this
usb device, thus android denies us access to it.

To me the money shot was actually the information from the libdivecomputer log where
dc_serial_open() in serial_posix.c was being called. This appeared to indicated that it was trying to
open a device named "ftdi" vs doing the check for the fake name and handling it
through the other usb code.
I'm assuming that the permission issue is that it could not open a file named "ftdi" because it doesn't have write permission to the directory to create a file by that name
rather than an actual usb permission issue since the incorrectly called serial code was trying to open a device named "ftdi" which does not exist and is not even a device.

--- bill

Anton Lundin

2018-09-16 11:52:36 UTC

Post by Bill Perry
The android app downloaded from the Playstore behaves differently than the one I just built from current sources.

How did you build the mobile app for android?

note: did you see the latest strangeness: http://lists.subsurface-divelog.org/pipermail/subsurface/2018-September/033025.html
Where libdivecomputer command/message delays don't work on the 4.8.1 app from the playstore but do work with the 4.8.1 that I built.
This was verified with a logic analyzer with the apps running on a Galaxy S4 with Android 4.4.4 and a Galaxy S7 running Android 8.0.0
I followed the instructions from the INSTALL file, except the only package I added to a newly installed Mint 18 OS install was cmake.
I'm building on freshly installed fully up to date 64bit Linux Mint MATE 18.3 in a virtual box vm.
I forked the subsurface repository then cloned from my github repo.
I did the submodule update to get the proper libdivecomputer
I then ran bash subsurface/packaging/android/android-build-wrapper.sh to build the .apk
I never built the code for the desktop.
After I sent the first post about this, I went back and started over with clean clone of the repo and
I installed all the recommended Ubuntu 18.04 packages (even though Mint 18 is based on Ubuntu 16.04)
and rebuilt the app after a clean/fresh clone.
That build would come up but immediately crash on both Android 4.4.4 and Android 8.0.0
I then started over again with a clean/fresh clone and checked out v4.8.1 and then
followed the same steps to build the apk
Nothing else was changed on the machine other then I started with a clean clone and checked out v4.8.1 vs using the head of Master.
This apk worked as expected including having the proper command/message delays
whereas the current app from the app store (4.8.1) does not have the proper delays.
I'm at a loss to explain the delay issue, but I can see it on a logic analyzer.

Might be anything. My suspect would be powersave on your android device.

Nope. This has nothing to do with each other.

I hear you and agree it shouldn't be related but I saw this more than once.
I wasn't focusing on this at the time so I just went ahead and connected to the cloud account so I
could get the serial port to spit out some data to look at the timing.
I'll go back and try to replicate and verify this and the details on if/how to replicate it
assuming I can replicate it.
Also, there is no way to download using the FTDI cable if Bluetooth is not enabled.
If bluetooth is not enabled, it won't let you download at all.

Ok. Just another bug. Document how to reproduce it, and file it.

You haven't built the app properly.

Doesn't matter. That libusb isn't used for anything, and if it does
matter, triangulate it and file the bug.

Post by Bill Perry
The big clue for me is that dc_serial_open() is being called.
This only gets called for /dev type devices.
It should not be called for the fake/magic "ftdi" device filename that is used when selecting the "FTDI" device
from the download UI.

Wrong. Read the code. Its always called as a last resort.

This is the money shot: You didn't give Subsurface mobile access to this
usb device, thus android denies us access to it.

Wrong. Read the code. The ftdi_open call is before dc_serial_open. If
ftdi_open fails, then dc_serial_open is called.

//Anton

ps. I think the conditional return in:
#ifdef SERIAL_FTDI
if (!strcmp(data->devname, "ftdi")) {
rc = ftdi_open(&data->iostream, context);
if (rc == DC_STATUS_SUCCESS)
return rc;
}
#endif

is wrong. If the devname is "ftdi" we should always return, whatever
ftdi_open returned.

ds.

--
Anton Lundin +46702-161604

Bill Perry

2018-09-16 15:48:22 UTC

Post by Bill Perry
Nothing else was changed on the machine other then I started with a clean clone and checked out v4.8.1 vs using the head of Master.
This apk worked as expected including having the proper command/message delays
whereas the current app from the app store (4.8.1) does not have the proper delays.
I'm at a loss to explain the delay issue, but I can see it on a logic analyzer.
Might be anything. My suspect would be powersave on your android device.

Turns out I'm blind and couldn't read my own notes.
The playstore app was 4.8.0 not 4.8.1
The delay issue has been resolved between the 4.8.0 release and the 4.8.1 release.
Total f'up on my part. I deserved to be lashed for this.
So sorry for wasting peoples time on this.

Post by Bill Perry
Also, there is no way to download using the FTDI cable if Bluetooth is not enabled.
If bluetooth is not enabled, it won't let you download at all.

Ok. Just another bug. Document how to reproduce it, and file it.

Will do.

Post by Bill Perry
Wrong. Read the code. The ftdi_open call is before dc_serial_open. If
ftdi_open fails, then dc_serial_open is called.
//Anton
#ifdef SERIAL_FTDI
if (!strcmp(data->devname, "ftdi")) {
rc = ftdi_open(&data->iostream, context);
if (rc == DC_STATUS_SUCCESS)
return rc;
}
#endif
is wrong. If the devname is "ftdi" we should always return, whatever
ftdi_open returned.
ds.

Ah. I overlooked that. That does seem wrong.

Dirk Hohndel

2018-09-16 16:35:42 UTC

Turns out I'm blind and couldn't read my own notes.
The playstore app was 4.8.0 not 4.8.1
The delay issue has been resolved between the 4.8.0 release and the 4.8.1 release.

Darn - I should have updated the playstore app. Well, I'll do this in the next couple of days with 4.8.2/2.1.2

Post by Bill Perry
#ifdef SERIAL_FTDI
if (!strcmp(data->devname, "ftdi")) {
rc = ftdi_open(&data->iostream, context);
if (rc == DC_STATUS_SUCCESS)
return rc;
}
#endif
is wrong. If the devname is "ftdi" we should always return, whatever
ftdi_open returned.

Ah. I overlooked that. That does seem wrong.

Yep, I pulled Anton's fix this morning, that will also be fixed in the upcoming version 2.1.2
of Subsurface-mobile

/D

Bill Perry

2018-09-16 18:18:52 UTC