Silly problem with dive sites and GPS downloading

[Linus Torvalds]

So here's a patch that fixes a problem with the Garmin GPS downloading,
but I'm not entirely sure what the _real_ fix is.

Let me explain the background first.

Each dive has a dive site, and the dive site is identified with an uuid.
Now, I happen to think that's a horrible design, but it does mean that you
can just pick a dive site for a dive, and then if you have ten different
dives on the same dive site, you can edit the dive site, and the dive site
information will be changed for all ten dives, because you're editing not
the dive, but the information about the site.

In contrast, what we *used* to do was to just put the dive site data into
the dive itself, and if you edited the dive site, it would only change for
that one dive. You could still pick the same dive site for another dive,
but all that would do was to copy the dive site data at the time you
picked it - if you fixed the GPS coordinates (or the name) later, it would
change only for the particular dive you edited.

So there's a very real reason we do that indirection, and the uuid has all
kinds of theoretical advantages (if we actually had a better model of
managing it), but the uuid has actually caused a huge amount of problems,
and in my opinion it has created more problems than it ever was supposed
to solve.

As an example of one of the problems it created (long ago) was that if you
imported the same dive data twice, you'd get different uuid's for the dive
site data, and then you'd get really nasty conflicts when you merged
things. For example, maybe in one case you just had the name, but then in
the other import you did a better job and added GPS location too, but now
both copies of your dive has a dive site, and it's entirely unclear which
is the better data.

Our merge strategy (when you have dives on two different machines, because
you use a laptop _and_ a cellphone, for example) is *not* semantically
aware, because that would be a nightmare, it's just a line-by-line git
merge. Which completely breaks down when you have two lines that are
different just because they have two different uuid's on them.

So things like uuids are fundamentally bad for things like that.

The way we fixed the nastiest of the merge problem was to just say "ok,
the uuid isn't actually random when we create it, it's calculated as a
hash of the dive site name and the time of the dive". That didn't "fix"
the problem, but at least it meant that if you imported the same dive data
twice from an external source, you at least got the exact same uuids and
didn't get nasty merge conflicts just because you happened to import it
twice.

HOWEVER.

We now have a new problem exactly *because* we don't actually use a random
uuid, but generate it based on divetime and name. It's related to the fact
that the libdivecomputer download can now create new dive sites
automatically if the dive computer supports GPS data for the dive. Right
now that only affects the Garmin Descent Mk1, but I really hope other dive
computers will follow suit in the future, because the GPS location really
is very very convenient, and it's by far my favorite feature of the
Descent Mk1 (which is otherwise a fairly "Meh" dive computer).

What happens is that you download your dives for the day, and you are
reall yhappy to just get the location data automatically. But the Garmin
doesn't know the _name_ of the dive site, obviously, it just has the GPS,
so you edit the name, and you're all done. Great. No problem.

Now, you're on a dive vacation, and the end of the *next* day, you
download the new dives for _that_ day, and you get all the GPS data for
the new dives too. Everything is fine, right?

No, not everything is fine. Because what happens is that the
libdivecomputer download doesn't just download the new dives, it starts
downloading _all_ the dives from the Garmin Descent, and parses them, and
in the process eventually notices "Oh, I already had this dive", and
that's when it stops downloading.

That still sounds fine, right? You never see the old dive, because we
noticed it was old, so it doesn't matter.

Wrong again. As part of parsing the dive download, it obviously parses the
GPS data, and it generates the dive site information for the GPS data.
And this happens REGARDLESS of whether the downloaded dive is actually
used or not.

And, in fact, because we use the same name, and the same dive time, we are
guaranteed to create the same dive site uuid when we do this. See above
about *why* we very intentionally do this. So when we download a dive -
whether we will actually *use* that dive later or not - we will be filling
in the dive site information with the data we got from the dive computer.

... and in the process we will be overwriting the data that we filled in
manually yesterday. The name of the dive site, but also possibly even the
GPS of the dive site (maybe the user decided to edit that using the map,
because while the automatically downloaded GPS data was "correct", maybe
the user wanted to change it to be the actual under-water location using
the satellite data, rather than the place where you started the dive or
where you surfaced.

There are a few obvious solutions to this mess:

a) the uuid approach and indirection just isn't worth it.

b) just make the libdivecomputer download not use the dive time, but
"time of download" of something for the dive site time, and thus
effectively generate a new uuid for every download.

c) notice when we already have a pre-existing matching uuid, and just use
the old information for the newly downloaded dive.

Honestly, (a) has been my opinion for years now. It's a pain. We've never
had a good dive site editing model. The indirection has theoretical
advantages that we simply don't take advantage of, and it has real
disadvantages that have shown up many times.

However, while I'd personally prefer (a), it's not actually practical. We
have a lot of UI code built up around it that I'm not going to touch, and
maybe some day we can actually make use of the dive site indirection.
There is *some* small existing advantage too (that whole "shared dive site
description" thing), and people probably use it.

So (a) is out of the picture, I just wanted to mention it because it would
be my "in a perfect world" solution.

(b) is what the patch below actually implements. It will cause merge
conflicts if you download the same dive on two different machines without
having done a cloud sync in between, but honestly, you'll probably get
those merge conflicts anyway, and it shouldn't be fatal.

(c) is the other approach that would make sense, but the way the dive site
code is organized, it's just simply a more painful patch. It might be the
better approach, though.

I've added my sign-off to the patch, although the above explanation should
be made into a commit log entry to explain it.

Comments?

Linus

PS. We have another pending problem with the dive site situaiton: the
Garmin Descent Mk1 gives us both entry and exit coordinates, and having
done four drift dives with it, I actually really *would* like to have our
mapping to show it as not a flag, but as a line between two points. But
right now we only associate a single GPS coordinate with the dive, and
because the Garmin reliably gives an exit coordinate but not an entry one
(if you jump into the water before it gets a GPS fix), the downloader will
just use the single exit point for the automatic dive site.

But that pending problem is an enhancement, not a bug.

---

Signed-off-by: Linus Torvalds <***@linux-foundation.org>

core/libdivecomputer.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/core/libdivecomputer.c b/core/libdivecomputer.c
index 2b535dfaf..fcc774425 100644
--- a/core/libdivecomputer.c
+++ b/core/libdivecomputer.c
@@ -12,6 +12,7 @@
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
+#include <time.h>
#include "gettext.h"
#include "dive.h"
#include "subsurface-string.h"
@@ -615,7 +616,7 @@ static void parse_string_field(struct dive *dive, dc_field_string_t *str)
longitude = parse_degrees(line, &line);

if (latitude.udeg && longitude.udeg)
- dive->dive_site_uuid = create_dive_site_with_gps(str->value, latitude, longitude, dive->when);
+ dive->dive_site_uuid = create_dive_site_with_gps(str->value, latitude, longitude, time(NULL));
}
}
#endif

[Dirk Hohndel]

Post by Linus Torvalds
What happens is that you download your dives for the day, and you are
reall yhappy to just get the location data automatically. But the Garmin
doesn't know the _name_ of the dive site, obviously, it just has the GPS,
so you edit the name, and you're all done. Great. No problem.
Now, you're on a dive vacation, and the end of the *next* day, you
download the new dives for _that_ day, and you get all the GPS data for
the new dives too. Everything is fine, right?
No, not everything is fine. Because what happens is that the
libdivecomputer download doesn't just download the new dives, it starts
downloading _all_ the dives from the Garmin Descent, and parses them, and
in the process eventually notices "Oh, I already had this dive", and
that's when it stops downloading.
That still sounds fine, right? You never see the old dive, because we
noticed it was old, so it doesn't matter.
Wrong again. As part of parsing the dive download, it obviously parses the
GPS data, and it generates the dive site information for the GPS data.
And this happens REGARDLESS of whether the downloaded dive is actually
used or not.

now THAT is the actual bug, IMHO.
We should only keep dive sites that are referenced from dives that we keep.
This isn't a case that we've had before since as you correctly point out we didn't
use to create dive sites when downloading from a dive computer, but regardless,
a dive site that was created as we parse a dive which we then ignore should also
be ignored.

Post by Linus Torvalds
And, in fact, because we use the same name, and the same dive time, we are
guaranteed to create the same dive site uuid when we do this. See above
about *why* we very intentionally do this. So when we download a dive -
whether we will actually *use* that dive later or not - we will be filling
in the dive site information with the data we got from the dive computer.
... and in the process we will be overwriting the data that we filled in
manually yesterday. The name of the dive site, but also possibly even the
GPS of the dive site (maybe the user decided to edit that using the map,
because while the automatically downloaded GPS data was "correct", maybe
the user wanted to change it to be the actual under-water location using
the satellite data, rather than the place where you started the dive or
where you surfaced.
a) the uuid approach and indirection just isn't worth it.
b) just make the libdivecomputer download not use the dive time, but
"time of download" of something for the dive site time, and thus
effectively generate a new uuid for every download.
c) notice when we already have a pre-existing matching uuid, and just use
the old information for the newly downloaded dive.
(b) is what the patch below actually implements. It will cause merge
conflicts if you download the same dive on two different machines without
having done a cloud sync in between, but honestly, you'll probably get
those merge conflicts anyway, and it shouldn't be fatal.
(c) is the other approach that would make sense, but the way the dive site
code is organized, it's just simply a more painful patch. It might be the
better approach, though.

Or my suggestion (d) to not create a dive site when not accepting the dive.

But yeah, I'm ok with (b) for now.

Post by Linus Torvalds
I've added my sign-off to the patch, although the above explanation should
be made into a commit log entry to explain it.

I'll apply this to master

Post by Linus Torvalds
PS. We have another pending problem with the dive site situaiton: the
Garmin Descent Mk1 gives us both entry and exit coordinates, and having
done four drift dives with it, I actually really *would* like to have our
mapping to show it as not a flag, but as a line between two points. But
right now we only associate a single GPS coordinate with the dive, and
because the Garmin reliably gives an exit coordinate but not an entry one
(if you jump into the water before it gets a GPS fix), the downloader will
just use the single exit point for the automatic dive site.
But that pending problem is an enhancement, not a bug.

Yes, and we already had people who said that they want us to associate a
path with the dive. And my answer to this continues to be that the visualization
is a pain and that I am not convinced that the accuracy of these GPS coordinates
is nowhere near good enough to make all this worth it.

/D

[Linus Torvalds]

Post by Dirk Hohndel

Post by Linus Torvalds
Wrong again. As part of parsing the dive download, it obviously parses the
GPS data, and it generates the dive site information for the GPS data.
And this happens REGARDLESS of whether the downloaded dive is actually
used or not.

now THAT is the actual bug, IMHO.
We should only keep dive sites that are referenced from dives that we keep.

I agree.

But this is all a direct result of the completely broken UUID model we have.

So what you are suggesting is what I suggested too: that (a)
alternative that gets rid of UUID's.

I'd love to. They are broken garbage. But it's not an option for the
reasons I already outlined.

Post by Dirk Hohndel
This isn't a case that we've had before since as you correctly point out we didn't
use to create dive sites when downloading from a dive computer, but regardless,
a dive site that was created as we parse a dive which we then ignore should also
be ignored.

But YOU CANNOT.

Seriously.

Let me explain again.

(a) importing a dive from a dive computer _fundamentally_ creates a
"struct dive". That's what it does. You don't "select a dive" to be
downloaded later. You create "struct dive" entities, and then you
select which of those you use.

(b) a dive doesn't *have* dive site information. It doesn't even have
a pointer to a dive site. All it has is that COMPLETELY AND UTTERLY
BROKEN "uuid" that I dislike so much.

(c) thus, in order to parse location information, you *have* to not
only create a dive site, you have to create a UUID, so that you can
associate the dive with the dive site information.

This is all a _direct_ result of that whole crazy "dive site uuid"
model. It is wrong. I didn't write it. I detest that code.

But it's basically unfixable as it is now. As long as a "struct dive"
contains that broken "uint32_t dive_site_uuid;" as the dive site
descriptor, we *have* to create the dive site this way.

Now, if the dive site is then never exposed (because the dive was
never accepted), it won't be *saved* when we save the final XML or git
format, so it will eventually be thrown away. But it has to be
created, and a uuid has to be allocated for it, as long as we have
that broken uuid model for dive sites.

I'd love for somebody to do my suggested (a) and just get rid of the
crazy uuid concept, and just make dive sites be a pointer from the
"struct dive".

But that's a lot of UI code, and all the crazy divesite handling code
that I had nothing to do with, so the person who does that needs to be
somebody else than me, I strongly suspect.

Linus

[Linus Torvalds]

On Sat, Sep 22, 2018 at 2:35 PM Linus Torvalds

Post by Linus Torvalds
But it's basically unfixable as it is now. As long as a "struct dive"
contains that broken "uint32_t dive_site_uuid;" as the dive site
descriptor, we *have* to create the dive site this way.

Side note: one possible solution is to get rid of that dive_site_uuid,
and replace it with a "struct divesite *ds" instead.

Then, all the internal code is made to just use the pointers to the
"struct divesite".

The only code that would use the uuid is the loading and saving code.
The saving code would create a uuid by just hashing all the dive site
data (so name, gps, information), and thus create a fake "uuid" that
is really just a placeholder for the actual data. The loading code
would create the divesite, and then use the uuid to look it up, but
we'd never actually have to deal with a uuid in any of the code that
*uses* divesites.

That would solve a lot of problems.

But there is a lot of UI code that uses that uuid right now. It
doesn't deal in "struct divesite *", it literally deals in those
uuid's.

That's the main problem with fixing this. I could do the loading and
saving part. But not the Qt model parts..

Linus

[Dirk Hohndel]

Post by Linus Torvalds
On Sat, Sep 22, 2018 at 2:35 PM Linus Torvalds

Post by Linus Torvalds
But it's basically unfixable as it is now. As long as a "struct dive"
contains that broken "uint32_t dive_site_uuid;" as the dive site
descriptor, we *have* to create the dive site this way.

Side note: one possible solution is to get rid of that dive_site_uuid,
and replace it with a "struct divesite *ds" instead.
Then, all the internal code is made to just use the pointers to the
"struct divesite".
The only code that would use the uuid is the loading and saving code.
The saving code would create a uuid by just hashing all the dive site
data (so name, gps, information), and thus create a fake "uuid" that
is really just a placeholder for the actual data. The loading code
would create the divesite, and then use the uuid to look it up, but
we'd never actually have to deal with a uuid in any of the code that
*uses* divesites.
That would solve a lot of problems.
But there is a lot of UI code that uses that uuid right now. It
doesn't deal in "struct divesite *", it literally deals in those
uuid's.
That's the main problem with fixing this. I could do the loading and
saving part. But not the Qt model parts..

I'm currently knee deep in the mobile UI... if I stop I will lose hours
of understanding what goes where... so let me finish that (ha!) and
then I'll think through this and see if I can come up with a way to
make this work in the UI.

I'm not ignoring your idea - I'm just not able to task switch :-)

/D

[Berthold Stoeger]

Hi Linus,

Post by Linus Torvalds
On Sat, Sep 22, 2018 at 2:35 PM Linus Torvalds

Post by Linus Torvalds
But it's basically unfixable as it is now. As long as a "struct dive"
contains that broken "uint32_t dive_site_uuid;" as the dive site
descriptor, we *have* to create the dive site this way.

Side note: one possible solution is to get rid of that dive_site_uuid,
and replace it with a "struct divesite *ds" instead.

This is actually on my TODO-list, notably when making import-dives undo-able.
Just like on undo/redo of add-dives we have to remove/add implicitly generated
dive-trips, on undo/redo of import-dives we'll have to remove/add new dive-
sites.

I don't see the UI part as a problem. Doing away with the "uniq-id" of dives
on desktop was quite easy and I think the same is true for dive-site uuids.

Not sure you want to wait that long, though.

Berthold

[Anton Lundin]

Post by Berthold Stoeger
Hi Linus,

Post by Linus Torvalds
On Sat, Sep 22, 2018 at 2:35 PM Linus Torvalds

Post by Linus Torvalds
But it's basically unfixable as it is now. As long as a "struct dive"
contains that broken "uint32_t dive_site_uuid;" as the dive site
descriptor, we *have* to create the dive site this way.

Side note: one possible solution is to get rid of that dive_site_uuid,
and replace it with a "struct divesite *ds" instead.

This is actually on my TODO-list, notably when making import-dives undo-able.
Just like on undo/redo of add-dives we have to remove/add implicitly generated
dive-trips, on undo/redo of import-dives we'll have to remove/add new dive-
sites.
I don't see the UI part as a problem. Doing away with the "uniq-id" of dives
on desktop was quite easy and I think the same is true for dive-site uuids.
Not sure you want to wait that long, though.

My first thought when I read Linus issues I thought this problem would
be solvable via your undo work.


Maybe we can focus on getting rid of dive_site_uuid first, to solve
this specific issue, and look at the general undo later.


Another thought was that the DLF-import tingie also just blindly creates
dive sites when it sees a gps coordinate. Any issues we have with the
Garmin download, is probably the same there.


Quite a bit OT, but continuing in that tangent, it might be time for the
DLF thingie to grow up and become a proper libdivecomputer backend,
instead of the subsurface specific importer we got right now. All the
usb-file-import infrastructure got created for the Garmin downloader.



//Anton

[Dirk Hohndel]

Post by Anton Lundin
Quite a bit OT, but continuing in that tangent, it might be time for the
DLF thingie to grow up and become a proper libdivecomputer backend,
instead of the subsurface specific importer we got right now. All the
usb-file-import infrastructure got created for the Garmin downloader.

I was actually thinking that maybe a couple of our current file system
based importers could reasonable migrate into libdivecomputer now.
I haven't spent enough time to think this through in more detail, but
the more we can homogenize the experience, the better for our users.

/D

[Linus Torvalds]

On Sat, Sep 22, 2018 at 11:33 PM Berthold Stoeger

Post by Berthold Stoeger
This is actually on my TODO-list, notably when making import-dives undo-able.
Just like on undo/redo of add-dives we have to remove/add implicitly generated
dive-trips, on undo/redo of import-dives we'll have to remove/add new dive-
sites.

Yes, makes sense.

Post by Berthold Stoeger
I don't see the UI part as a problem. Doing away with the "uniq-id" of dives
on desktop was quite easy and I think the same is true for dive-site uuids.

Very good. I think you're a lot more comfortable with the C++ UI code,
so if you think it's not a big deal, then that's great.

Post by Berthold Stoeger
Not sure you want to wait that long, though.

Right now this is a *tiny* issue, so it's absolutely not critical.
It's more of a small annoyance than anything else.

So if you're planning on addressing it at some point, then I wouldn't
worry about the wait.

Linus

[Monty Taylor]

Post by Dirk Hohndel

Post by Linus Torvalds
PS. We have another pending problem with the dive site situaiton: the
Garmin Descent Mk1 gives us both entry and exit coordinates, and having
done four drift dives with it, I actually really *would* like to have our
mapping to show it as not a flag, but as a line between two points. But
right now we only associate a single GPS coordinate with the dive, and
because the Garmin reliably gives an exit coordinate but not an entry one
(if you jump into the water before it gets a GPS fix), the downloader will
just use the single exit point for the automatic dive site.
But that pending problem is an enhancement, not a bug.

Yes, and we already had people who said that they want us to associate a
path with the dive. And my answer to this continues to be that the visualization
is a pain and that I am not convinced that the accuracy of these GPS coordinates
is nowhere near good enough to make all this worth it.

The visualization of a path would certainly be a pain, but having start
and end coordinates for a dive would certainly be neat - and for folks
without an Mk1 who are using the gps logger, we can grab an ending GPS
coordinate with time matching just as easily as we can grab a beginning one.

Maybe at least collect the start/end GPS so we have them in the data,
and maybe later someone will good idea for visualization?

[Linus Torvalds]

Post by Monty Taylor
Maybe at least collect the start/end GPS so we have them in the data,
and maybe later someone will good idea for visualization?

For the Garmin Descent, the way it's currently done is that start/end
coordinates are captures as "extra data" strings, with a key of GPS1
and GPS2 respectively.

End result: the data does get saved, but only the last coordinate is
then used for the dive site.

So I agree, there's no huge hurry about this.

Looking at the actual data I do have, it does seem to be (a) more than
precise enough to warrant showing on the map and (b) not useful as a
_path_.

For example, I did Blackrock in Maui as a "drift" dive (ok, so it took
an hour and a half to "drift" a few hundred meters because there was
no real current), and for that dive I got

keyvalue "GPS1" "20.928930, -156.695058"
keyvalue "GPS2" "20.926903, -156.696113"

which if you look at a map looks exactly right, but if you draw a line
between them it will go straight through the Sheraton Maui, because
obviously the actual dive is *around* the rock.

So I think Dirk's argument that we don't have good enough GPS location
is wrong - but it is true that it might be hard to show them sanely.

I think the Garmin Connect app showed the locations as a red and a
green marker. I'm not sure that's great either.

Linus

[Dirk Hohndel]

So Linus said some interesting things about this topic - and others
have added in the past. I'd like to try and connect some of the dots
here - please correct me if I get things wrong...

(a) a dive site, as an independent entity from a dive, does have a
logical GPS location. One can argue where that is and in many ways
that's a matter of taste and opinion (e.g., is it where you enter the water,
or is it where the "interesting" part of the dive happens), but in general
a dive site has just one coordinate.

(b) a dive itself can be described in multiple ways.
(1) Simply by the coordinates of the site.
(2) by the entry and exit point.
(3) by the path that the diver actually took (turning the whole dive profile
into a 3D path).

Today we do (b)(1).
It seems that at least with the Garmin we could relatively easily (assuming
the diver does turn on dive mode early enough to get a GPS fix before
being under water) do (b)(2)
I don't think there is equipment that is widely available to do (b)(3)

Typically when it's hard to foresee how things will get abstracted out
int the future, I tend to suggest using text fields / strings. Right now we
store all available GPS information on the Garmin as strings. Maybe
we should allow people to do this for other dive computers as well,
assuming they have a source for the strings (and we can of course use
the GPS info from a phone). Which means we'd need a way to do this
dive computer independently.

I'd love to hear people's thoughts on this.

/D

Post by Linus Torvalds

Post by Monty Taylor
Maybe at least collect the start/end GPS so we have them in the data,
and maybe later someone will good idea for visualization?

For the Garmin Descent, the way it's currently done is that start/end
coordinates are captures as "extra data" strings, with a key of GPS1
and GPS2 respectively.
End result: the data does get saved, but only the last coordinate is
then used for the dive site.
So I agree, there's no huge hurry about this.
Looking at the actual data I do have, it does seem to be (a) more than
precise enough to warrant showing on the map and (b) not useful as a
_path_.
For example, I did Blackrock in Maui as a "drift" dive (ok, so it took
an hour and a half to "drift" a few hundred meters because there was
no real current), and for that dive I got
keyvalue "GPS1" "20.928930, -156.695058"
keyvalue "GPS2" "20.926903, -156.696113"
which if you look at a map looks exactly right, but if you draw a line
between them it will go straight through the Sheraton Maui, because
obviously the actual dive is *around* the rock.
So I think Dirk's argument that we don't have good enough GPS location
is wrong - but it is true that it might be hard to show them sanely.
I think the Garmin Connect app showed the locations as a red and a
green marker. I'm not sure that's great either.
Linus
_______________________________________________
subsurface mailing list
http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface

[Linus Torvalds]

Post by Dirk Hohndel
(a) a dive site, as an independent entity from a dive, does have a
logical GPS location. One can argue where that is and in many ways
that's a matter of taste and opinion (e.g., is it where you enter the water,
or is it where the "interesting" part of the dive happens), but in general
a dive site has just one coordinate.

I'd actually argue that a dive site could be described as an "area",
not a single point, but I think your argument that a "dive" and a
"divesite" are entirely separate things is correct.

Post by Dirk Hohndel
(b) a dive itself can be described in multiple ways.
(1) Simply by the coordinates of the site.
(2) by the entry and exit point.
(3) by the path that the diver actually took (turning the whole dive profile
into a 3D path).
Today we do (b)(1).

Yes.

Except I'd go even further, and say that we could have a fourth case:
a dive could be described as simply a location, *WITHOUT* having a
dive site associated with it.

Because I'd be a lot happier about our divesite handling if we didn't
force a dive to be associated with a site.

I think a dive site exists independently of any dives. A dive site it
about having a location, a description, and a name, but is *not*
defined by you diving it.

And similarly, you can have a dive without having a "site". Even if
the dive has a single location, that location may not be a "site".

The most common case of that "non-site" case is drift-dives, where you
might drift between possibly several sites (or drift into or out of
one).

But people literally do blackwater dives too, where you absolutely do
*not* have a site at all. You're literally just in a fairly random
location. You're doing it for

So what I think would be better is that we separate the dive sites
from the location further.

For example, I'd love to have a dive site database, but I'd love that
database to have *NOTHING* to do with the dives I do. It would be
literally used only to look up names based on the GPS coordinates I
have.

If subsurface would take my GPS coordinates, and automatically name
the location of my dives, that would be really cool. I could still
edit the name (without editing the dive site database), because many
of the places I dive, there's a buoy that can take me to several
different areas just depending on which direction I swim.

So maybe the divesite database would be "Molokini Crater" for stuff in
a certain GPS range, and I'd get that automatically just based on the
GPS info I have (either from the subsurface phone app tracking, or
from a dive computer like the Garmin that has gps).

But then I might edit it to say "Enenu'i" or "Reef's End" or
"Aquarium" depending on which part of Molokini I was diving (ok, so
the GPS might be able to separate Enenu'i from Reef's End, but when
moored at the Reef's end site, you do tend to go off in different
directions and the "sites" have different names).

So I think it would be much nicer if we just made the dive location be
per-dive, and stopped associating dives with "sites" entirely. Having
some nice way to get to a site based on "this is close to your dive"
would be great, but I think it shouldn't be tied together the way it
is now.

Then, if we had a separate dive site database, and you *don't* have
GPS location, you could just select a site when you edit the dive, and
that would *copy* the gps information from the site database to the
dive, but it wouldn't make that hard link (again, you might then edit
the per-dive data, maybe by just moving the marker around - without
that having any effect what-so-ever on the divesite database).

The problem, of course, is that it does require that dive site
database editor. Which we've never had. Because we mixed up the dive
location editing with the divesite database.

But separating out the divesite database would be good for another
reason: if we ever were to have some external "cloud database" of dive
sites that people can share, it obviously needs to be separate from
peoples dives anyway.

I think we could easily do the separation part, but the difficult part
is doing that divesite editor (even if it is entirely private, the
"cloud database" is just a pipedream right now, and has been for
years)

Linus

[Linus Torvalds]

On Mon, Sep 24, 2018 at 9:33 AM Linus Torvalds

Post by Linus Torvalds
But people literally do blackwater dives too, where you absolutely do
*not* have a site at all. You're literally just in a fairly random
location. You're doing it for

Oops, this got cut short because I was editing something else.

It was supposed to be "You're doing it for the sealife, not the
particular location".

Now, blackwater dives tend to be about plankton etc, but you do have
other similar cases where you follow the life, not some geographic
feature. We *tried* to find whale-sharks. Again, had that worked
better, it wouldn't necessarily be anywhere _near_ a "dive site", it
would be "hey, we saw a whale shark and jumped in at this location".

Linus

[Monty Taylor]

Post by Linus Torvalds

Post by Dirk Hohndel
(a) a dive site, as an independent entity from a dive, does have a
logical GPS location. One can argue where that is and in many ways
that's a matter of taste and opinion (e.g., is it where you enter the water,
or is it where the "interesting" part of the dive happens), but in general
a dive site has just one coordinate.

I'd actually argue that a dive site could be described as an "area",
not a single point, but I think your argument that a "dive" and a
"divesite" are entirely separate things is correct.

Post by Dirk Hohndel
(b) a dive itself can be described in multiple ways.
(1) Simply by the coordinates of the site.
(2) by the entry and exit point.
(3) by the path that the diver actually took (turning the whole dive profile
into a 3D path).
Today we do (b)(1).

Yes.
a dive could be described as simply a location, *WITHOUT* having a
dive site associated with it.
Because I'd be a lot happier about our divesite handling if we didn't
force a dive to be associated with a site.

++

Post by Linus Torvalds
I think a dive site exists independently of any dives. A dive site it
about having a location, a description, and a name, but is *not*
defined by you diving it.
And similarly, you can have a dive without having a "site". Even if
the dive has a single location, that location may not be a "site".
The most common case of that "non-site" case is drift-dives, where you
might drift between possibly several sites (or drift into or out of
one).
But people literally do blackwater dives too, where you absolutely do
*not* have a site at all. You're literally just in a fairly random
location. You're doing it for
So what I think would be better is that we separate the dive sites
from the location further.
For example, I'd love to have a dive site database, but I'd love that
database to have *NOTHING* to do with the dives I do. It would be
literally used only to look up names based on the GPS coordinates I
have.
If subsurface would take my GPS coordinates, and automatically name
the location of my dives, that would be really cool. I could still
edit the name (without editing the dive site database), because many
of the places I dive, there's a buoy that can take me to several
different areas just depending on which direction I swim.
So maybe the divesite database would be "Molokini Crater" for stuff in
a certain GPS range, and I'd get that automatically just based on the
GPS info I have (either from the subsurface phone app tracking, or
from a dive computer like the Garmin that has gps).
But then I might edit it to say "Enenu'i" or "Reef's End" or
"Aquarium" depending on which part of Molokini I was diving (ok, so
the GPS might be able to separate Enenu'i from Reef's End, but when
moored at the Reef's end site, you do tend to go off in different
directions and the "sites" have different names).
So I think it would be much nicer if we just made the dive location be
per-dive, and stopped associating dives with "sites" entirely. Having
some nice way to get to a site based on "this is close to your dive"
would be great, but I think it shouldn't be tied together the way it
is now.
Then, if we had a separate dive site database, and you *don't* have
GPS location, you could just select a site when you edit the dive, and
that would *copy* the gps information from the site database to the
dive, but it wouldn't make that hard link (again, you might then edit
the per-dive data, maybe by just moving the marker around - without
that having any effect what-so-ever on the divesite database).

Yeah - I think that's right on. It handles the simple case "I dove on
these coordinates" Then, if you WANT to create a dive site in the dive
site database, or in the magical future world of ponies and rainbows you
want to associate it with a pre-existing dive site from a magical
dive-site database, you totally can.

Post by Linus Torvalds
The problem, of course, is that it does require that dive site
database editor. Which we've never had. Because we mixed up the dive
location editing with the divesite database.
But separating out the divesite database would be good for another
reason: if we ever were to have some external "cloud database" of dive
sites that people can share, it obviously needs to be separate from
peoples dives anyway.
I think we could easily do the separation part, but the difficult part
is doing that divesite editor (even if it is entirely private, the
"cloud database" is just a pipedream right now, and has been for
years)

I like the idea of a dive site database/editor and would be happy to
work on such a thing.

[Dirk Hohndel]

Post by Linus Torvalds
I'd actually argue that a dive site could be described as an "area",

True. Because our data isn't confusing and inconsistent enough, we
could add a "focus point" plus NW and SE corner :-)

Post by Linus Torvalds

Post by Dirk Hohndel
(b) a dive itself can be described in multiple ways.
(1) Simply by the coordinates of the site.
(2) by the entry and exit point.
(3) by the path that the diver actually took (turning the whole dive profile
into a 3D path).
Today we do (b)(1).

Yes.
a dive could be described as simply a location, *WITHOUT* having a
dive site associated with it.
Because I'd be a lot happier about our divesite handling if we didn't
force a dive to be associated with a site.

But then it doesn't get a name. The moment you have a name, you
have implicitly defined a site.

Post by Linus Torvalds
I think a dive site exists independently of any dives. A dive site it
about having a location, a description, and a name, but is *not*
defined by you diving it.
And similarly, you can have a dive without having a "site". Even if
the dive has a single location, that location may not be a "site".

That part I disagree with. It may not be an "official site", it may not
match some random definition of "site", but a dive with a location
and name has happened at a spot that we should track as a dive site.

Post by Linus Torvalds
The most common case of that "non-site" case is drift-dives, where you
might drift between possibly several sites (or drift into or out of
one).

Yes, this goes back to the earlier argument that a dive site really should
have three coordinates, but I think we can agree that everyone considers
German Channel a dive site.

Post by Linus Torvalds
But people literally do blackwater dives too, where you absolutely do
*not* have a site at all. You're literally just in a fairly random
location. You're doing it for

Yes, this isn't a Site in the sense of a place that everyone comes back
to, yet your dive happened somewhere... and that's the site where you
went diving.

I think we are going to seriously confuse ourselves if we allow dives
to have location and name independent of a dive site. We used to do
that and as much as I know that you hate where it got us to, we moved
away from that for VERY GOOD REASONS.

Post by Linus Torvalds
So what I think would be better is that we separate the dive sites
from the location further.
For example, I'd love to have a dive site database, but I'd love that
database to have *NOTHING* to do with the dives I do. It would be
literally used only to look up names based on the GPS coordinates I
have.
If subsurface would take my GPS coordinates, and automatically name
the location of my dives, that would be really cool. I could still
edit the name (without editing the dive site database), because many
of the places I dive, there's a buoy that can take me to several
different areas just depending on which direction I swim.
So maybe the divesite database would be "Molokini Crater" for stuff in
a certain GPS range, and I'd get that automatically just based on the
GPS info I have (either from the subsurface phone app tracking, or
from a dive computer like the Garmin that has gps).
But then I might edit it to say "Enenu'i" or "Reef's End" or
"Aquarium" depending on which part of Molokini I was diving (ok, so
the GPS might be able to separate Enenu'i from Reef's End, but when
moored at the Reef's end site, you do tend to go off in different
directions and the "sites" have different names).
So I think it would be much nicer if we just made the dive location be
per-dive, and stopped associating dives with "sites" entirely. Having
some nice way to get to a site based on "this is close to your dive"
would be great, but I think it shouldn't be tied together the way it
is now.
Then, if we had a separate dive site database, and you *don't* have
GPS location, you could just select a site when you edit the dive, and
that would *copy* the gps information from the site database to the
dive, but it wouldn't make that hard link (again, you might then edit
the per-dive data, maybe by just moving the marker around - without
that having any effect what-so-ever on the divesite database).
The problem, of course, is that it does require that dive site
database editor. Which we've never had. Because we mixed up the dive
location editing with the divesite database.
But separating out the divesite database would be good for another
reason: if we ever were to have some external "cloud database" of dive
sites that people can share, it obviously needs to be separate from
peoples dives anyway.
I think we could easily do the separation part, but the difficult part
is doing that divesite editor (even if it is entirely private, the
"cloud database" is just a pipedream right now, and has been for
years)

That's a completely new definition of dive site. You are actually adding
a completely new semantic level. And that is based on how YOU think
about your diving and the places that you dive at.

And since this is how you think about it, it is not wrong. It is just completely
different from what we have today and will need a complete redesign of
how Subsurface operates, how we store data, how the user interaction
works, etc.

If others think that this is the direction we should go and want to
commit to writing the UI, clearly defining the semantic, mapping it
to what we have today, define how we migrate data, setup the
backend service that provides you with the GPS / name mapping,
document all this... that sounds like a great idea.

Another half-assed redefinition of what we do, how we do it,
how we store it, how it's displayed and edited by the user...
that I'm less in favor of.

/D

[Monty Taylor]

Post by Dirk Hohndel
So Linus said some interesting things about this topic - and others
have added in the past. I'd like to try and connect some of the dots
here - please correct me if I get things wrong...
(a) a dive site, as an independent entity from a dive, does have a
logical GPS location. One can argue where that is and in many ways
that's a matter of taste and opinion (e.g., is it where you enter the water,
or is it where the "interesting" part of the dive happens), but in general
a dive site has just one coordinate.
(b) a dive itself can be described in multiple ways.
(1) Simply by the coordinates of the site.
(2) by the entry and exit point.
(3) by the path that the diver actually took (turning the whole dive profile
into a 3D path).
Today we do (b)(1).
It seems that at least with the Garmin we could relatively easily (assuming
the diver does turn on dive mode early enough to get a GPS fix before
being under water) do (b)(2)
I don't think there is equipment that is widely available to do (b)(3)
Typically when it's hard to foresee how things will get abstracted out
int the future, I tend to suggest using text fields / strings. Right now we
store all available GPS information on the Garmin as strings. Maybe
we should allow people to do this for other dive computers as well,
assuming they have a source for the strings (and we can of course use
the GPS info from a phone). Which means we'd need a way to do this
dive computer independently.

Maybe also an option somewhere, for when the GPS data logger is being
used in the Android app, to save all of the collected GPS points as text
records onto the dive object as well when applying GPS data. (at least
the ones between start and end times of the dive)

Post by Dirk Hohndel
I'd love to hear people's thoughts on this.

Let me see if I can make it even more complicated - because I'm pretty
sure more input will definitely make this clearer.

A few of the other loggers, at least diviac and divelogs.de, split the
location of a dive into two ideas "Location" and "Dive Site". (neither
particularly explain the difference spectacularly) The way I've come to
understand the split is that Location is "the real geographical place
where I set up my gear" and Dive Site is "the place where I was under
water". Now with Mk1 GPS or other sets of coordinates, each dive also
potentially has a specific "path" - that is a path described across a
dive site.

Specific examples, to attach concrete examples to abstract concepts:

Blue Hole, NM:
https://www.google.com/maps/place/Blue+Hole/@34.9404462,-104.6819938,15z/data=!3m1!4b1!4m5!3m4!1s0x871c0b5ae1b0dfe9:0xe7228c5b0ba0698a!8m2!3d34.940447!4d-104.673239

Scuba Ranch, TX:
https://www.google.com/maps/place/The+Scuba+Ranch+at+Clear+Springs/@32.7891403,-96.1552064,17z/data=!3m1!4b1!4m5!3m4!1s0x86495b7f045849e7:0x7e45bec3eb6d9080!8m2!3d32.7891403!4d-96.1530177

Flynn Reef, Australia:
https://www.google.com/maps/place/Flynn+Reef/@-16.7243694,146.2558661,14z/data=!3m1!4b1!4m5!3m4!1s0x6979cbf27dcbb67d:0x778f3f250b640676!8m2!3d-16.7243701!4d146.2733758

Blue Hole, Scuba Ranch and Flynn Reef are all Locations.

Blue Hole has one eponymously named dive site and is super easy. There's
also only one entry/exit point. It's a hole.

Scuba Ranch is also just one "Location" - there is an entrance where you
pay a fee and where you can get your log stamped. There are maybe 10
different parking lots/shore entries (not sure the distinction matters,
but maybe it does?) - and in general different local groups have "their"
spot where they always dive from. Inside the flooded quarry there are
several "Dive Site"s - there's a sunken airplane, two boats, a big metal
fish and several dive platforms. If you dive there a bunch, maybe
because you're a local, which dive you did might be more interesting
than just "Scuba Ranch" - but also you're certainly at Scuba Ranch for
all of them.

Flynn Reef is a real geographic place with a real name, but being a reef
there are more than one spot on it where your boat might moor. From any
one of those moorings, there are multiple available "Dive Site"s ... and
I'm pretty sure the different dive operators call at least some of them
by different names.

Throwing the Linus Mk1 GPS start/end tracking in to the mix, and now
there are also the possibility of one or more distinct "Dive"s with
locations at a given "Dive Site".

Flynn Reef has GPS coordinates, it's a place. Nemo's Bommie also
probably has GPS coordinates - or more to the point is at least a
logical place where saying "these 4 dives were all at Nemo's Bommie"
makes sense as a human. Then each dive at that dive site has the
potential to also have specific GPS start/end coordinates, even though
the dive might be associated with a Dive Site and that Dive Site might
be associated with a Location.

As you go from General to Specific, the GPS information becomes less
likely to be shareable with others. I can _definitely_ tell you about
Blue Hole, NM, Scuba Ranch or Flynn Reef and you can _definitely_ go to
those places - and there isn't any dispute that they are places. The
Dive Sites at those places are also potentially interesting- but are
also up to interpretation in terms of what, if anything, to call them.
The specific Dive GPS coordinates are almost never interesting to
someone else, but still *might* be in case you're trying to tell someone
about a cool drift dive you did - and communicating "I went in at the
beach north of the Maui Sheraton then did a (slow) drift around Black
Rock and exited south of it" is maybe useful to track/communicate.

Sorry, that's a giant pile of words.

What I'd personally love to be able to do is arrive at a "Location",
pick up my phone/computer and say "I'm at a new Location called Flynn
Reef" or "I'm back at existing Location Flynn Reef". Then, as the
divemaster is giving the briefing (or if I'm giving myself the briefing)
I could pick up my phone and Add a new Dive Site called "Nemo's Bommie".
I could mark it as being at the Location Flynn Reef. I might want to
attach a map of the dive site to the Dive Site - and that map would be
relevant to any dives done there.

When I'm done with the dive, I import the dive and I could then select
"Nemo's Bommie" from the Dive Site list. If the dive came with its own
start/end GPS coordinates - or if I drift dove pulling a tender
overhead, maybe I left my phone with the GPS tracker running in the
tender so I've actually got a decent-ish path even. Maybe soon one of us
will get a https://www.navimate.com/ and

That's super complex - and it's possible people might not want to manage
3 different objects of informatoin each with 1:Many relationships. So if
we got more complex modelling of locations, it's probably important for
a human to be able to just dive, and say "I dove at this place" and not
care about having Location->DiveSite->Dive.

In some ways - it might be more accurate to consider Location and
DiveSite to both be areas rather than points. For instance, Flynn Reef
is actually a whole reef. Then, inside of that Area, there are multiple
Dive Sites, each of which being an area underwater that could be
described with a map. And then dives are paths on top of that under
water area.

In that sense, a Location is an area that is (or could be) a mappable
area by Google Maps. A Dive Site is an area under water, and a Dive is a
path. The simple cases are then that a given Location and Dive Site and
Dive are the same, and are a zero-dimensional area described by a single
GPS coordinate.

Or maybe that's WAY too much.

Post by Dirk Hohndel

Post by Linus Torvalds

Post by Monty Taylor
Maybe at least collect the start/end GPS so we have them in the data,
and maybe later someone will good idea for visualization?

For the Garmin Descent, the way it's currently done is that start/end
coordinates are captures as "extra data" strings, with a key of GPS1
and GPS2 respectively.
End result: the data does get saved, but only the last coordinate is
then used for the dive site.
So I agree, there's no huge hurry about this.
Looking at the actual data I do have, it does seem to be (a) more than
precise enough to warrant showing on the map and (b) not useful as a
_path_.
For example, I did Blackrock in Maui as a "drift" dive (ok, so it took
an hour and a half to "drift" a few hundred meters because there was
no real current), and for that dive I got
keyvalue "GPS1" "20.928930, -156.695058"
keyvalue "GPS2" "20.926903, -156.696113"
which if you look at a map looks exactly right, but if you draw a line
between them it will go straight through the Sheraton Maui, because
obviously the actual dive is *around* the rock.
So I think Dirk's argument that we don't have good enough GPS location
is wrong - but it is true that it might be hard to show them sanely.
I think the Garmin Connect app showed the locations as a red and a
green marker. I'm not sure that's great either.
Linus
_______________________________________________
subsurface mailing list
http://lists.subsurface-divelog.org/cgi-bin/mailman/listinfo/subsurface