Integrate with self-hostable fitness tracking service #49

Open
opened 7 years ago by cpfeiffer · 50 comments
Owner

The android application is probably not suited to record and keep, manage and visualize all fitness and sleep data, and what not.

Online services like Google Fit or Apple Health are obviously out of the question.

After digging a little I found these projects (in order of viability):

The android application is probably not suited to record and keep, manage and visualize all fitness and sleep data, and what not. Online services like Google Fit or Apple Health are obviously out of the question. After digging a little I found these projects (in order of viability): - https://wger.de (python, django, quite active, but tailored towards workout) - https://fluxtream.org/ https://github.com/fluxtream/fluxtream-app (service for collecting and managing all kinds of data, maybe a bit heavy for our needs, but still would be nice to support) - http://www.openmhealth.org/ health oriented service, https://github.com/openmhealth - http://giudici.us/work/communicating_health/ (research project, probably inactive, code is here: https://github.com/theguice/wearable-health) - https://github.com/jdjkelly/quant (ruby on rails, dormant for almost a year, but very much what we want) https://bitbucket.org/dalareo) - https://github.com/z1lc/qs (consists mostly of charts, unfortunately Google Charts) - https://github.com/selfhub/selfhub (no demo site?)wonder how self-hostable this is without a dedicated cloud provider like AWS, Azure, ...) - http://openfitapi.com/ (Drupal 7 extension, but dead since December 2012, code still available at
chrisvella commented 7 years ago (Migrated from github.com)
Owner

The other option would be to create something ourselves. I don't see why a html5 page wouldn't do the job. For the graphs http://www.chartjs.org/ looks really good.
I'm not sure how to we would get the values from the app as js variables. If that is worked out I may be able to provide assistance in creating the interface for the tool.

Pushing the data via git could be interesting. We could create a github page template that generates the graphs based on the contents of the file. Then anyone could easily host it for free, and easily replicate the setup by forking our repo.

If one of @cpfeiffer options is selected, making a openshift compatible quickstart would help adoption.

The other option would be to create something ourselves. I don't see why a html5 page wouldn't do the job. For the graphs http://www.chartjs.org/ looks really good. I'm not sure how to we would get the values from the app as js variables. If that is worked out I may be able to provide assistance in creating the interface for the tool. Pushing the data via git could be interesting. We could create a github page template that generates the graphs based on the contents of the file. Then anyone could easily host it for free, and easily replicate the setup by forking our repo. If one of @cpfeiffer options is selected, making a openshift compatible quickstart would help adoption.
Owner

@chrisvella
chartjs is what morpheuz uses (http://www.morpheuz.net)
Looks good to me also.

But I really disagree with @cpfeiffer here. I think an Android application IS suited to "record and keep, manage and visualize all fitness and sleep data, and what not."

We could take our data and render it locally in a webview without internet access if we really want to base this on html5.

If we add an "export to some place in the internet" option I would prefer writing a second app which complements GadgetBridge and has internet access to act as a forwarder between Gadgetbridge and the internet. Because adding internet access to Gadgetbridge is something I would like to avoid.

@chrisvella chartjs is what morpheuz uses (http://www.morpheuz.net) Looks good to me also. But I really disagree with @cpfeiffer here. I think an Android application IS suited to "record and keep, manage and visualize all fitness and sleep data, and what not." We could take our data and render it locally in a webview without internet access if we really want to base this on html5. If we add an "export to some place in the internet" option I would prefer writing a second app which complements GadgetBridge and has internet access to act as a forwarder between Gadgetbridge and the internet. Because adding internet access to Gadgetbridge is something I would like to avoid.
chrisvella commented 7 years ago (Migrated from github.com)
Owner

@ashimokawa I agree with you completely.
Interested to hear your thoughts @cpfeiffer

@ashimokawa I agree with you completely. Interested to hear your thoughts @cpfeiffer
Poster
Owner

Here's my thoughts:

  1. A mobile/handheld device is not suited for keeping/archiving data (be it fitness, sleep or any other data).
    Reasons:
    a) devices get lost, stolen, broken and backups are so unsexy that few people have them. So for those people that want to keep that data for more than a few days, an external location is better suited.
    b) some people may even have multiple mobile devices and track their activities with one or another, but want to have their data accumulated and consolidated in one place.
    c) the fitness guys and gals may want to share their data, have competitions and whatnot, maybe they want to annotate their data with notes, images, etc. I would rather have that in a separate (web-) application than Gadgetbridge (or a helper app).

  2. I completely agree with you that synchronizing with or exporting to a [self-hostable] web service should be optional in such a way that the required Android permissions (i.e. INTERNET) are optional, too. With current Android situation, this means that the export/sync support needs to be in a separately installable app -- which is perfectly fine for me.

  3. Having some local rendering support in Gadgetbridge is needed as well, of course. Just to have a look at "how well did I sleep last night", or to track fitness activity over a short time, say a few months or so.

Here's my thoughts: 1) A mobile/handheld device is not suited for keeping/archiving data (be it fitness, sleep or any other data). Reasons: a) devices get lost, stolen, broken and backups are so unsexy that few people have them. So for those people that want to keep that data for more than a few days, an _external_ location is better suited. b) some people may even have multiple mobile devices and track their activities with one or another, but want to have their data accumulated and consolidated in one place. c) the fitness guys and gals may want to share their data, have competitions and whatnot, maybe they want to annotate their data with notes, images, etc. I would rather have that in a separate (web-) application than Gadgetbridge (or a helper app). 2) I completely agree with you that synchronizing with or exporting to a [self-hostable] web service should be optional in such a way that the required Android permissions (i.e. INTERNET) are optional, too. With current Android situation, this means that the export/sync support needs to be in a separately installable app -- which is perfectly fine for me. 3) Having some local rendering support in Gadgetbridge is needed as well, of course. Just to have a look at "how well did I sleep last night", or to track fitness activity over a short time, say a few months or so.
chrisvella commented 6 years ago (Migrated from github.com)
Owner

It sounds like a html/js implementation of this would be a good compromise? That way we could put it on a self hosted service AND have a local view.

It sounds like a html/js implementation of this would be a good compromise? That way we could put it on a self hosted service AND have a local view.
Poster
Owner

If you ask me, yes, sharing code between the website and the local view would be beneficial. I think @ashimokawa prefers to do the charts himself though, because of much better performance. Let's see how it works out.

An html/js solution would definitely be a good thing, though, both server side and client side, e.g. for more sophisticated graphs.

If you ask me, yes, sharing code between the website and the local view would be beneficial. I think @ashimokawa prefers to do the charts himself though, because of much better performance. Let's see how it works out. An html/js solution would definitely be a good thing, though, both server side and client side, e.g. for more sophisticated graphs.
Owner

Actually using the android canvas is not more difficult than using html5 canvas.
But performance and - much more important memory to me - memory footprint is far better. The chromium based WebView since Android 4.4 likes to eat up a huge chunk of my 512MB.

Another problem I see is license of the js code we would have to include.

I am not against sharing js code if someone comes up with a good implementation and I can see the benefits and he/she explains how to deal with licenses of imported js code.

Actually using the android canvas is not more difficult than using html5 canvas. But performance and - much more important memory to me - memory footprint is far better. The chromium based WebView since Android 4.4 likes to eat up a huge chunk of my 512MB. Another problem I see is license of the js code we would have to include. I am not against sharing js code if someone comes up with a good implementation and I can see the benefits and he/she explains how to deal with licenses of imported js code.
Poster
Owner

Alternatively, we might use something like https://github.com/dodocat/MPAndroidChart or https://code.google.com/p/achartengine/ or https://github.com/blackfizz/EazeGraph although we would lose the benefit of sharing code with the hosted version.

But we certainly need a db schema before deciding on this.

Alternatively, we might use something like https://github.com/dodocat/MPAndroidChart or https://code.google.com/p/achartengine/ or https://github.com/blackfizz/EazeGraph although we would lose the benefit of sharing code with the hosted version. But we certainly need a db schema before deciding on this.
Owner

About data visualization we could also add something like this:
https://github.com/dlazaro66/WheelIndicatorView (NB: code quality / license unchecked!)
for the google-fit lovers (if any ;-) )

Btw for inspiration this community contains a lot of projects:
https://plus.google.com/communities/100609058582053363304
It's on google plus but can be accessed also without being registered/logged in.

About data visualization we could also add something like this: https://github.com/dlazaro66/WheelIndicatorView (NB: code quality / license unchecked!) for the google-fit lovers (if any ;-) ) Btw for inspiration this community contains a lot of projects: https://plus.google.com/communities/100609058582053363304 It's on google plus but can be accessed also without being registered/logged in.
Poster
Owner

Yeah, circles are hip :-)

Yeah, circles are hip :-)
Poster
Owner

FWIW, I've got a prototype displaying sleep and activity data with MPAndroidChart.

FWIW, I've got a prototype displaying sleep and activity data with MPAndroidChart.
Owner

Cool, looking forward to that ! :)

Cool, looking forward to that ! :)
1337sup3rh4x0r commented 6 years ago (Migrated from github.com)
Owner

me too!
Are you guys planning on rolling out released apks more frequently in the future or would it make sense for "interested noobs" like me to look into how to build from this source?

me too! Are you guys planning on rolling out released apks more frequently in the future or would it make sense for "interested noobs" like me to look into how to build from this source?
Owner

Yes, definitely. The next release will be out in a few days. And from then on we should release more often like it was the case in the beginning.
The "problem" is that we tried to tackle something big which is not yet in a releasable state but at the same time we had important bug fixes we want to have released.
My proposal is to release 0.4.0 on the weekend. That release will contain new and incomplete features which will improve and stabilize during the 0.4.x series.
0.2.0, 0.3.0 were the same. New features but stabilization took a few point releases.

Yes, definitely. The next release will be out in a few days. And from then on we should release more often like it was the case in the beginning. The "problem" is that we tried to tackle something big which is not yet in a releasable state but at the same time we had important bug fixes we want to have released. My proposal is to release 0.4.0 on the weekend. That release will contain new and incomplete features which will improve and stabilize during the 0.4.x series. 0.2.0, 0.3.0 were the same. New features but stabilization took a few point releases.
1337sup3rh4x0r commented 6 years ago (Migrated from github.com)
Owner

No need for stability from the start in this early stage. I'll be happy to fiddle around with anything you give me and report bugs. Really looking forward to wear the miband to track steps and sleep with the confidence that the data stays where it should!

No need for stability from the start in this early stage. I'll be happy to fiddle around with anything you give me and report bugs. Really looking forward to wear the miband to track steps and sleep with the confidence that the data stays where it should!
chrisvella commented 6 years ago (Migrated from github.com)
Owner

If CSV export was developed, we could explore something like: https://rohitrox.github.io/js_csv/
Users would be able to "host" the fitness tracking service on static hosts like github OR have it run offline with very little fuss.

Not sure how possible this is, but it would be super cool if users could fork a repo with the fitness tracker (pretty sure github lets you host static sites pretty easily). Then configure from the app a destination for the CSV file (could we have it committed straight to the forked repo?). Then the fitness tracker would automatically load the file.

@ashimokawa take a look at this: http://www.androidpolice.com/2015/06/06/android-m-will-never-ask-users-for-permission-to-use-the-internet-and-thats-probably-okay/

If CSV export was developed, we could explore something like: https://rohitrox.github.io/js_csv/ Users would be able to "host" the fitness tracking service on static hosts like github OR have it run offline with very little fuss. Not sure how possible this is, but it would be super cool if users could fork a repo with the fitness tracker (pretty sure github lets you host static sites pretty easily). Then configure from the app a destination for the CSV file (could we have it committed straight to the forked repo?). Then the fitness tracker would automatically load the file. @ashimokawa take a look at this: http://www.androidpolice.com/2015/06/06/android-m-will-never-ask-users-for-permission-to-use-the-internet-and-thats-probably-okay/
Poster
Owner

@chrisvella Yes, that would be pretty cool!

@chrisvella Yes, that would be pretty cool!
rolandgeider commented 6 years ago (Migrated from github.com)
Owner

Hi! I just found this issue by accident, I'm the main developer of wger. While my app's focus are workouts, I'd be interested what other data you guys would be logging and graphing, perhaps I can get some ideas ;)

Hi! I just found this issue by accident, I'm the main developer of wger. While my app's focus are workouts, I'd be interested what other data you guys would be logging and graphing, perhaps I can get some ideas ;)
Poster
Owner

Hi Roland, cool to have you here :-)
we haven't made any progress or even decision on this issue yet, because we have lots to do.

Those activity trackers are primarily used for walking, AFAIU (being a non-walker myself -- I prefer other sports). Those devices typically count 'steps', but have some support for other sports activities. I don't know whether we'll ever be able to support other activities, though because that would probably mean interpreting raw sensor data which would involve much more time and knowhow than we currently have.

Another interesting type of 'activity' is 'sleep'. The devices track your daily sleep, not only in hours, but also devided into light sleep and deep sleep.

So, to summarize: our primary needs of supported activities are:

  1. walking
  2. sleeping

Everything else is future stuff, unless someone comes up and helps implementing it.

PS: sorry that I summarized wger with such few words, I was actually amazed at what wger already provides

Hi Roland, cool to have you here :-) we haven't made any progress or even decision on this issue yet, because we have lots to do. Those activity trackers are primarily used for walking, AFAIU (being a non-walker myself -- I prefer other sports). Those devices typically count 'steps', but have some support for other sports activities. I don't know whether we'll ever be able to support other activities, though because that would probably mean interpreting raw sensor data which would involve much more time and knowhow than we currently have. Another interesting type of 'activity' is 'sleep'. The devices track your daily sleep, not only in hours, but also devided into light sleep and deep sleep. So, to summarize: our primary needs of supported activities are: 1. walking 2. sleeping Everything else is future stuff, unless someone comes up and helps implementing it. PS: sorry that I summarized wger with such few words, I was actually amazed at what wger already provides
rolandgeider commented 6 years ago (Migrated from github.com)
Owner

Hey!

Thanks for the kind words, but my project still has a long way to go! :) I had already thought (shortly) about adding some kind of sleeping log, but was unsure on how to do it or whether it would be useful at all if the data has to be entered manually. Should I implement this, I'll let you know

Hey! Thanks for the kind words, but my project still has a long way to go! :) I had already thought (shortly) about adding some kind of sleeping log, but was unsure on how to do it or whether it would be useful at all if the data has to be entered manually. Should I implement this, I'll let you know
Poster
Owner

But the workout data has to be entered manually as well, no?
We could provide daily automatically generated sleep and steps data :-)

I'm very interested!

But the workout data has to be entered manually as well, no? We could provide daily automatically generated sleep and steps data :-) I'm very interested!
rolandgeider commented 6 years ago (Migrated from github.com)
Owner

Yes they have to be entered by hand, but those are more tangible, since you enter them as you go or right after the workout. Sleep is more difficult without a gadget (how many hours did I really sleep? and the light and deep phases are out of the question).

Another possibility would be to implement on my side only REST API endpoints (i.e. without a GUI) for the sleep and perhaps even the step data.

Yes they have to be entered by hand, but those are more tangible, since you enter them as you go or right after the workout. Sleep is more difficult without a gadget (how many hours did I really sleep? and the light and deep phases are out of the question). Another possibility would be to implement on my side only REST API endpoints (i.e. without a GUI) for the sleep and perhaps even the step data.
Poster
Owner

I certainly agree that one wouldn't want to add such data by hand -- I wouldn't want to do it with other activity data either :-)

And yes, an API endpoint would be enough for users to upload their data.

Is wger somehow customizable in what types of activities shall be available? While we can interface with many different systems, I think we should also promote a default web service which is focused on what Gadgetbridge supports; which is sleep and walking (steps) at the moment.

I certainly agree that one wouldn't want to add such data by hand -- I wouldn't want to do it with other activity data either :-) And yes, an API endpoint would be enough for users to upload their data. Is wger somehow customizable in what types of activities shall be available? While we can interface with many different systems, I think we should also promote a default web service which is focused on what Gadgetbridge supports; which is sleep and walking (steps) at the moment.
janvonde commented 6 years ago (Migrated from github.com)
Owner

I discovered this project yesterday and it is exactly what I was looking for for a long time. Using a cool gadget like the Mi Band collecting personal information but without the need to upload it to any company.

I would really love to have a simple self hosted solution where I can push my data. I am thinking about a small PHP script that receives the data via POST and stores it on a sqlite/mysql database. You could secure the transport if you serve the website via HTTPS. You could secure your data if you'd put a basic authentication in front of it.

The app would need to support the following:

  • configure optional full URL to the self hosted PHP script
  • accept valid https connections
  • support basic authentication
  • send data via POST to this URL

The data could be send a JSON object for example.

I am willing to help the project, I have very limited PHP scripting skills but I could setup a sample site and script that does exactly this.

I discovered this project yesterday and it is exactly what I was looking for for a long time. Using a cool gadget like the Mi Band collecting personal information but without the need to upload it to any company. I would really love to have a simple self hosted solution where I can push my data. I am thinking about a small PHP script that receives the data via POST and stores it on a sqlite/mysql database. You could secure the transport if you serve the website via HTTPS. You could secure your data if you'd put a basic authentication in front of it. The app would need to support the following: - configure optional full URL to the self hosted PHP script - accept valid https connections - support basic authentication - send data via POST to this URL The data could be send a JSON object for example. I am willing to help the project, I have very limited PHP scripting skills but I could setup a sample site and script that does exactly this.
chrisvella commented 6 years ago (Migrated from github.com)
Owner

@janvonde FYI gadgetbridge supports exporting as a sqlite file. If you plan on developing something, it might be an idea to work on importing an sqlite file for now.

I had a look into a html5 way of working with gadgetbridge exporting. I have tested it out, sql.js is able to import gadgetbridge files. https://github.com/kripken/sql.js

@janvonde FYI gadgetbridge supports exporting as a sqlite file. If you plan on developing something, it might be an idea to work on importing an sqlite file for now. I had a look into a html5 way of working with gadgetbridge exporting. I have tested it out, sql.js is able to import gadgetbridge files. https://github.com/kripken/sql.js
Poster
Owner

@janvonde Yes, that would be awesome. Do you only want to get hold of the data and later see what you can do with it? Or do you want to have statistics, graphs and possibly adding of additional data in the php script?

As @chrisvella pointed out, right now you can export the entire sqlite database and do all kinds of things with it (see the 'Debug' activity). Maybe a CSV export would be more helpful than the entire database, though (see #67).

In any case -- help in that area is more than welcome :-)

@janvonde Yes, that would be awesome. Do you only want to get hold of the data and later see what you can do with it? Or do you want to have statistics, graphs and possibly adding of additional data in the php script? As @chrisvella pointed out, right now you can export the entire sqlite database and do all kinds of things with it (see the 'Debug' activity). Maybe a CSV export would be more helpful than the entire database, though (see #67). In any case -- help in that area is more than welcome :-)
Poster
Owner

Updated the order of the researched projects.

Updated the order of the researched projects.
janvonde commented 6 years ago (Migrated from github.com)
Owner

@cpfeiffer I want to receive the data on my own server, store it in a database and see how I can visualize it there. I only want to visualize existing data that I receive, no manual input or other things are in my head so far. Simple and stupid but secure and hopefully some nice graphs ;)

@cpfeiffer I want to receive the data on my own server, store it in a database and see how I can visualize it there. I only want to visualize existing data that I receive, no manual input or other things are in my head so far. Simple and stupid but secure and hopefully some nice graphs ;)
Poster
Owner

@janvonde That's the perfect start. Maybe other data will come later, e.g. annotating what you've done some day, others might want to record their weight (loss), etc. But yeah, please start small :-)

@janvonde That's the perfect start. Maybe other data will come later, e.g. annotating what you've done some day, others might want to record their weight (loss), etc. But yeah, please start small :-)
janvonde commented 6 years ago (Migrated from github.com)
Owner

How can I export data as sqlite?

How can I export data as sqlite?
Poster
Owner

Use the debug activity and press export database. The db will then be the gadgetbridge directory on your SD card.

Use the debug activity and press export database. The db will then be the gadgetbridge directory on your SD card.
janvonde commented 6 years ago (Migrated from github.com)
Owner

Is this a released feature already available at f-droid?

Is this a released feature already available at f-droid?
Poster
Owner

No, apparently it's not released yet, sorry. It will be in 0.4.4.

No, apparently it's not released yet, sorry. It will be in 0.4.4.
janvonde commented 6 years ago (Migrated from github.com)
Owner

Thanks for the information @cpfeiffer . I will keen an eye on this issue. Once 0.4.4 is released I will look at the database. In the meantime I set up a vhost with https and basic authentication already. I put three lines of php code to make sure I am proper receiving JSON objects via post. I will look now for a basic user interface. Once I've seen the database structure including some example data I can make a propose on how the JSON object could look like. After you've seen the first result (don't expect something to big or beautiful) you can decide if you would like to have the code somewhere here as well. But I will publish it anyway 8-)

Thanks for the information @cpfeiffer . I will keen an eye on this issue. Once 0.4.4 is released I will look at the database. In the meantime I set up a vhost with https and basic authentication already. I put three lines of php code to make sure I am proper receiving JSON objects via post. I will look now for a basic user interface. Once I've seen the database structure including some example data I can make a propose on how the JSON object could look like. After you've seen the first result (don't expect something to big or beautiful) you can decide if you would like to have the code somewhere here as well. But I will publish it anyway 8-)
Poster
Owner

Maybe you want to have a look at how the other systems listed in this issue do it, i.e. how the API looks like. Being somehow compatible to those, maybe even to Google Fit might be a good thing. I would rather see our data adapted to the needs of the other systems than have other systems directly use the values as found in our database (which might change at some point).

After you've seen the first result (don't expect something to big or beautiful) you can decide if you would like to have the code somewhere here as well. But I will publish it anyway 8-)

Promised? :-)

Maybe you want to have a look at how the other systems listed in this issue do it, i.e. how the API looks like. Being somehow compatible to those, maybe even to Google Fit might be a good thing. I would rather see our data adapted to the needs of the other systems than have other systems directly use the values as found in our database (which might change at some point). > After you've seen the first result (don't expect something to big or beautiful) you can decide if you would like to have the code somewhere here as well. But I will publish it anyway 8-) Promised? :-)
rolandgeider commented 6 years ago (Migrated from github.com)
Owner

Sorry for the late reply!

None of the logging in wger is customazible in the sense that each type has its own specific table, REST endpoint, etc. and adding a new type is not just making some configurations. While I'm not sure I can add them in the near future, I'd like to know a bit about the data structures. Is each step logged individually or are they aggregated every $time_interval? Do they also have geographic information? Or some kind of intensity/acceleration? (You can also just point me to a model file)

A different thing I wanted to mention here, since I keep reading about HTML and graphs: this [1] JS library that sits on top of D3js (the gold standard on JS graph libraries) and focuses on making time based graphs easy, which is what is being logged here. I plan on using it in wger and I'm just waiting for it to support one additional feature I need. Compared to manually using D3 it's really like day and night.

[1] http://metricsgraphicsjs.org/

Sorry for the late reply! None of the logging in wger is customazible in the sense that each type has its own specific table, REST endpoint, etc. and adding a new type is not just making some configurations. While I'm not sure I can add them in the near future, I'd like to know a bit about the data structures. Is each step logged individually or are they aggregated every $time_interval? Do they also have geographic information? Or some kind of intensity/acceleration? (You can also just point me to a model file) A different thing I wanted to mention here, since I keep reading about HTML and graphs: this [1] JS library that sits on top of D3js (the gold standard on JS graph libraries) and focuses on making time based graphs easy, which is what is being logged here. I plan on using it in wger and I'm just waiting for it to support one additional feature I need. Compared to manually using D3 it's really like day and night. [1] http://metricsgraphicsjs.org/
Poster
Owner

@rolandgeider No problem, we all have lots to do...

These are good questions! The devices play a role in this. The Mi Band provides one sample of activity data per minute and both step count and intensity.

The pebble does not provide this out of the box at all, you need a custom watch app that does this and so it's up to the watch app developer. With the morpheuz watch app it is one sample aggregated per 20min, in the near future we will get one sample per 10min.

Geographic information is not available, but could be added by Gadgetbridge if wanted (you would need to carry both the mobile and the device, though).

I haven't yet looked at what other activity tracking sites have as data model.

Thanks a lot for the pointer to metricgraphics, this looks awesome!

@rolandgeider No problem, we all have lots to do... These are good questions! The devices play a role in this. The Mi Band provides one sample of activity data per minute and both step count and intensity. The pebble does not provide this out of the box at all, you need a custom watch app that does this and so it's up to the watch app developer. With the morpheuz watch app it is one sample aggregated per 20min, in the near future we will get one sample per 10min. Geographic information is not available, but could be added by Gadgetbridge if wanted (you would need to carry both the mobile and the device, though). I haven't yet looked at what other activity tracking sites have as data model. Thanks a lot for the pointer to metricgraphics, this looks awesome!
simonvanderveldt commented 6 years ago (Migrated from github.com)
Owner

I discovered this project yesterday and it is exactly what I was looking for for a long time. Using a cool gadget like the Mi Band collecting personal information but without the need to upload it to any company.

I would really love to have a simple self hosted solution where I can push my data.

Wanted to say thanks for the cool app as well and add my +1 for gadgetbridge to function as a simple bridge ;) as proposed by @janvonde

It would only have to support the following:

  • Configurable endpoint
  • JSON formatted data
  • Push only new data

But I really disagree with @cpfeiffer here. I think an Android application IS suited to "record and keep, manage and visualize all fitness and sleep data, and what not."

Also wanted to say I really disagree with this ;) Keep the functionality of the app focussed, i.e. make it the best bridge app possible.
If visualisation is needed simply create a separate app for that or delegate it to a self-hosted web app.

Also @cpfeiffer open mHealth seems like a good option.
I'll probably try to get open mHealth, fluxtream and quant running in august to see how they compare.

> I discovered this project yesterday and it is exactly what I was looking for for a long time. Using a cool gadget like the Mi Band collecting personal information but without the need to upload it to any company. > > I would really love to have a simple self hosted solution where I can push my data. Wanted to say thanks for the cool app as well and add my +1 for gadgetbridge to function as a simple bridge ;) as proposed by @janvonde It would only have to support the following: - Configurable endpoint - JSON formatted data - Push only new data > But I really disagree with @cpfeiffer here. I think an Android application IS suited to "record and keep, manage and visualize all fitness and sleep data, and what not." Also wanted to say I really disagree with this ;) Keep the functionality of the app focussed, i.e. make it the best bridge app possible. If visualisation is needed simply create a separate app for that or delegate it to a self-hosted web app. Also @cpfeiffer [open mHealth](http://www.openmhealth.org/) seems like a good option. I'll probably try to get open mHealth, fluxtream and quant running in august to see how they compare.
Poster
Owner

@simonvanderveldt Thanks for the pointer to open mHealth, I added it to the list. Looks good to me.

And please let us know about your findings with the services you testdrive, this will help us a lot!

@simonvanderveldt Thanks for the pointer to open mHealth, I added it to the list. Looks good to me. And please let us know about your findings with the services you testdrive, this will help us a lot!
animalillo commented 6 years ago (Migrated from github.com)
Owner

This would be a lovely feature, so you can check your progress from your pc, and no need to take the phone for everything, plus bigger screen, better and bigger graphs... endless possibilities.

I love this app, but right now is missing a lot of what i expect from the xiaomi band, based on what the official app offers.

I'm a professional php developer, a bit busy right now, but wouldnt mind making a rest interface for the data storage based on json.

Maybe by the end of this month or during the next one i take a look at the android database and try and make a php interface for the app.

But for making something would be good to know what to expect and where is the project going.

Also, regarding the internet permissions, I dont mind either way, as the app (or both apps) are OpenSource, it could be bundled on the same app, or included in an external optional api app, i dont really mind this point.

This would be a lovely feature, so you can check your progress from your pc, and no need to take the phone for everything, plus bigger screen, better and bigger graphs... endless possibilities. I love this app, but right now is missing a lot of what i expect from the xiaomi band, based on what the official app offers. I'm a professional php developer, a bit busy right now, but wouldnt mind making a rest interface for the data storage based on json. Maybe by the end of this month or during the next one i take a look at the android database and try and make a php interface for the app. But for making something would be good to know what to expect and where is the project going. Also, regarding the internet permissions, I dont mind either way, as the app (or both apps) are OpenSource, it could be bundled on the same app, or included in an external optional api app, i dont really mind this point.
Poster
Owner

I love this app, but right now is missing a lot of what i expect from the xiaomi band, based on what the official app offers.

We know and we're working on it. It's just.. a lot of work, so it takes time. But we have most of the boilerplate code in place, so (visible) progress in accelerating, and it's quite easy to add new features.

I'm a professional php developer, a bit busy right now, but wouldnt mind making a rest interface for the data storage based on json.

Excellent, exactly what we need :-) We can create a git repository for it and give you all permissions, if that would be OK with you. Maybe you even want to team up with @janvonde.

But for making something would be good to know what to expect and where is the project going.

See the list of issues at https://github.com/Freeyourgadget/Gadgetbridge/issues/ for an overview. It doesn't mean that we can really do all of that, but we would like to if we find the resources.

For a start, we want to support

  • tracking sleep activity
  • tracking sports activity
  • notifications on the device

We're already support much of this, with some missing pieces and missing polish.

> I love this app, but right now is missing a lot of what i expect from the xiaomi band, based on what the official app offers. We know and we're working on it. It's just.. a lot of work, so it takes time. But we have most of the boilerplate code in place, so (visible) progress in accelerating, and it's quite easy to add new features. > I'm a professional php developer, a bit busy right now, but wouldnt mind making a rest interface for the data storage based on json. Excellent, exactly what we need :-) We can create a git repository for it and give you all permissions, if that would be OK with you. Maybe you even want to team up with @janvonde. > But for making something would be good to know what to expect and where is the project going. See the list of issues at https://github.com/Freeyourgadget/Gadgetbridge/issues/ for an overview. It doesn't mean that we can really do all of that, but we would like to if we find the resources. For a start, we want to support - tracking sleep activity - tracking sports activity - notifications on the device We're already support much of this, with some missing pieces and missing polish.
simonvanderveldt commented 6 years ago (Migrated from github.com)
Owner

And please let us know about your findings with the services you testdrive, this will help us a lot!

@cpfeiffer I've looked at a couple of the items in the list:

  • Open mHealth: Very loosely coupled components + lack of documentation made this a difficult one to get going. But, the major issue is there's no integrated dashboard/way to view your stats. I'll post a question on their mailing list about this to see if they have some suggestions.
  • fluxtream: looked at the setup instructions and then decided that that's way too difficult for 90% of the people
  • Found a new project Reportr and tried it out. Was relatively easy to setup, but the charts are fairly limited and for some reason it only showed counts for measurements instead of actual measurements for me.

So, unfortunately nothing sticks out as being a good solution :(

I'm using InfluxDB together with Grafana for the (time series) data from my Home Automation system and guess I'll just use that for my Gadgetbridge/Quantified Self needs as well...

> And please let us know about your findings with the services you testdrive, this will help us a lot! @cpfeiffer I've looked at a couple of the items in the list: - Open mHealth: Very loosely coupled components + lack of documentation made this a difficult one to get going. But, the major issue is there's no integrated dashboard/way to view your stats. I'll post a question on their mailing list about this to see if they have some suggestions. - fluxtream: looked at the [setup instructions](https://fluxtream.atlassian.net/wiki/display/FLX/Installation%3A+Linux) and then decided that that's way too difficult for 90% of the people - Found a new project [Reportr](https://github.com/Reportr/dashboard) and tried it out. Was relatively easy to setup, but the charts are fairly limited and for some reason it only showed counts for measurements instead of actual measurements for me. So, unfortunately nothing sticks out as being a good solution :( I'm using [InfluxDB](https://influxdb.com/) together with [Grafana](http://grafana.org/) for the (time series) data from my Home Automation system and guess I'll just use that for my Gadgetbridge/Quantified Self needs as well...
Poster
Owner

@simonvanderveldt Thanks for reporting back, much appreciated.

InfluxDB and Grafana look indeed nice. if you have something to share, even if just a report how you set this up or a screenshot, please do :-)

@simonvanderveldt Thanks for reporting back, much appreciated. InfluxDB and Grafana look indeed nice. if you have something to share, even if just a report how you set this up or a screenshot, please do :-)
simonvanderveldt commented 6 years ago (Migrated from github.com)
Owner

I tried quant today. Again basically no documentation on how to build/run it, but managed to make it work. It needs a postgres db to store all measurements, which makes it a bit of a hassle, but doable.

In general it covers most of what people will need, but the views/dashboards are limited to how they come out of the box, there is no way to create your own dashboard from the UI.
Included views/dashboards are list below with behind them the values that are shown in cards on the dashboard:

  • Weight (weight, fat%, lean mass, date, calculates BMI)
  • Meals (Calories, Fat, Carbs, Protein, Description)
  • Sleeps (from and to, calculates duration, earliest start last 7 days, recording streak and health sleeps count for last 14 days)
  • Journal Entries (Happiness, Feelings, Strategies)
  • An overview containing all of the above cards

All of the views contain the above mentioned cards + a tabular view of all the data. There are no charts or further historical views of the data.

So what's missing is a dedicated steps/activity view + charts and history.
Also I haven't looked at how to get the data in automatically (though you can enter data via web UI which are POSTS to the API, so that shouldn't be a problem).

Out of the options I've tried I'd say quant is the best option, but it will need some work to make it usable (adding history/charts + steps/activity).

I personally don't like being limited to predefined dashboards, so I'll stick with my current idea of Influxdb + Grafana.

@cpfeiffer Once I manage to get some data in from Gadgetbridge I'll post an update here with info + a screenshot

I tried quant today. Again basically no documentation on how to build/run it, but managed to make it work. It needs a postgres db to store all measurements, which makes it a bit of a hassle, but doable. In general it covers most of what people will need, but the views/dashboards are limited to how they come out of the box, there is no way to create your own dashboard from the UI. Included views/dashboards are list below with behind them the values that are shown in cards on the dashboard: - Weight (weight, fat%, lean mass, date, calculates BMI) - Meals (Calories, Fat, Carbs, Protein, Description) - Sleeps (from and to, calculates duration, earliest start last 7 days, recording streak and health sleeps count for last 14 days) - Journal Entries (Happiness, Feelings, Strategies) - An overview containing all of the above cards All of the views contain the above mentioned cards + a tabular view of all the data. There are no charts or further historical views of the data. So what's missing is a dedicated steps/activity view + charts and history. Also I haven't looked at how to get the data in automatically (though you can enter data via web UI which are POSTS to the API, so that shouldn't be a problem). Out of the options I've tried I'd say quant is the best option, but it will need some work to make it usable (adding history/charts + steps/activity). I personally don't like being limited to predefined dashboards, so I'll stick with my current idea of Influxdb + Grafana. @cpfeiffer Once I manage to get some data in from Gadgetbridge I'll post an update here with info + a screenshot
Owner

A new (to me) service: https://github.com/openmhealth/shimmer.

Nevermind, it was already in the list on the first post. Sorry for the noise.

A new (to me) service: https://github.com/openmhealth/shimmer. Nevermind, it was already in the list on the first post. Sorry for the noise.
bigretromike commented 6 years ago (Migrated from github.com)
Owner

Hey,

Is there a roadmap for this ? or is it - in day far far away ?
Why is "Online services like Google Fit or Apple Health are obviously out of the question." ?
I see that alot of people that will force to use gadgetbridge and still want the Google Fit sync will self-host anything that will be only as link to google_fit_botnet.
Why not make it posible to use it directly ?

Hey, Is there a roadmap for this ? or is it - in day far far away ? Why is "Online services like Google Fit or Apple Health are obviously out of the question." ? I see that alot of people that will force to use gadgetbridge and still want the Google Fit sync will self-host anything that will be only as link to google_fit_botnet. Why not make it posible to use it directly ?
Poster
Owner

@bigretromike Good point, we should also support syncing to proprietary services (in whatever way). Could you please open a separate ticket for this?

@bigretromike Good point, we should also support syncing to proprietary services (in whatever way). Could you please open a separate ticket for this?
Owner

ups I just closed an issue about this. I'll reopen it

## ups I just closed an issue about this. I'll reopen it
Owner

This javascript port of sqlite seems to work well:
http://kripken.github.io/sql.js/GUI/

In fact I just loaded my database (freshly exported from GB) and run queries on it. So perhaps someone is willing to implement a nice dashboard/gui on top of our database ?

This javascript port of sqlite seems to work well: http://kripken.github.io/sql.js/GUI/ In fact I just loaded my database (freshly exported from GB) and run queries on it. So perhaps someone is willing to implement a nice dashboard/gui on top of our database ?
vanous added the
activity post processing
activity/health
labels 12 months ago
vanous added the
network companion app
label 9 months ago
Owner
This can now be taken to another level.... https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/
Sign in to join this conversation.
No Milestone
No Assignees
4 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.