Agreed! This is another task in our huge backlog that we’ve been hoping to get around to eventually. Maybe once custom tracking is out :) I agree it should make a difference in predicting how behaviours affect one another across more than one day.

This would be amazing!

His would be huge! So many things affect you the next day, but even just knowing how the previous night’s sleep affects your day is worth the effort.

This is the biggest thing for me for sure. Got only 5hrs sleep last night, or haven’t worked out all week? Doesn’t matter how much I exercise or what I ate today Going to be a terrible day.

This would be the biggest thing for me too!

Maybe this would be a separate feature request/suggestion, could this go beyond the previous day? Maybe like a few days back? I ask because I’ve heard of cases where there is a delayed allergic reaction to foods and I’ve personally had this happen with a bee sting. Sometimes it’s not obvious to go back and connect it with something that happened, for example, 3 days before. And I imagine the use-case could extend beyond food.

I’d love to see this as well! Tho I don’t really need the app to tell me that when I have a lot of drinks the day before then I have a slow/unproductive day the next day. I would be curious to see if there’s a correlation between active time and productivity, or between coffee/alcohol and sleep quality.

It might be interesting to call this something like “deep analysis” and just have it running when the load on the server is low. Then surprise people with longer term correlations, but focused around the more useful use cases described above versus just going after every possible correlation (which would cut down on noise and save processor time).

Of course if the correlation was too difficult to make, or user’s questions became too annoying, then the answer could come back with error code 42… (See Hitchhiker’s Guide to the Galaxy: Deep Thought for more on this).

I believe you should do a proper cross correlation on the time-series data. Such that you can look for peaks in the cross correlation result at any time offset, not just at 0 and +/- 1 day.

Because correlations are pre-computed this isn’t really possible, and off the top of my head I don’t see the value in correlating daily values with large offsets. “You get more sleep when you walked more 5 days ago”? I suspect any results would be fairly spurious.

I think a lot of them probably would be spurious, but for example, tracking migraine triggers usually involves up to a 72 hour offset (e.g. sleeping in might result in a migraine up to 3 days later) so in some cases it could be really useful as a long-term pattern.

Ah, that’s an interesting example!

I know very little about the technical side of the correlations on this, so take my thoughts with a few grains of salt…

If the cumulative effects are common, then these are perhaps worthy of automation /codification, and only the large-offsets of particular significance, like the migraine example could be handled as one-offs specialized searches to avoid a flood of random correlations with no meaning (I.e. noise)…

I wonder if having a way to “deprioritize” a particular correlation that comes out of the engine would be a useful approach to cutting down noise. Say the engine finds a correlation between getting steps and a higher heart rate (seems a bit redundent) – then it would just tuck the correlation down at the bottom of the page and not include it in any emails, etc…

My bigger thought with all of this is to ensure the usefulness of any new features increase the balance of useful information and cuts down on noise that would lead to people not using the tool. All of us in here may have a bit higher tolerances for noise as we are avid enough users to be thinking and responding to feature ideas here…

Yes there will be a way to flag correlations as not useful and hide them.

Further to @John Eder’s comment –

Not only would deprioritization be handy, we should be able to opt-in to tracking across previous days per each custom tag/metric.

Eg: ☑️ previous day correlation – worked late and today correlation – productivity metric

That way we don’t have to go through all of the correlations that pop up for previous days and remove the ones we don’t want (number of emails sent yesterday + steps today) – but rather track the ones we do.

similar to multiple day correlation idea, but simpler implementation: correlation of today with the running average from previous seven days. i know my mood starts to drop if i haven’t exercised/coded/slept well in aggregate in the last few days, you know what i mean?

Just adding one more thumbs up to the previous ones, I’m tracking all kinds of sleep data, this would be a huge addition to find correlations.

Can’t wait for this to be finalised!

Done. Correlations get updated once a week, on Sundays, so as of next week you’ll see some correlations across two days :) Hope people find them useful!

Thanks a lot Josh!

Josh. Can you give a little better explanation of how this will work? Are you saying that correlations are now done over the same day and the day prior?

I’ve planned to write up an announcement with some details, but yes, that’s the gist of it. Until now, correlations compared data across two attributes for the same days, giving you things like “You walk more when you’re less productive”, but this update means we’ll also compare attributes across two days, making possible correlations like “You walk more the day after you’re more productive”.

Thanks! Looking forward to it!

Oooh, that’s great! Thank you. Looking forward to seeing what pops up next week.

Wow, this is amazing, can’t wait to see what new correlations will pop up! Thanks!

Thanks Josh! Looking forward to next week

Since this was introduced, it seems ALL DATA (slow) for viewing trends have removed. Does that mean that we don’t actually get to see “all data” anymore?

Yes, because correlations use up to 1 year of data now, there’s no way to generate correlations across all data. This isn’t to say that you can’t “see” the data, you can use the historical browsing in the Dashboard, plus the Trends pages, to see raw values and averages for all time. You just can’t have a single correlation calculated across multiple years of data.

Are plans to include a custom range for those in situations similar to the migrate example included in this?