Jekyll2021-03-27T11:42:02-07:00http://www.markwk.com/atom.xmlMark KoesterWriter, Product Designer and Software Engineer, based in Vence, California{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}A Year in Tasks, Projects and Goals: 20202021-03-16T00:00:00-07:002021-03-16T00:00:00-07:00http://www.markwk.com/year-in-tasks<p>My personal data analysis of my tasks reveals that I completed less tasks in 2020 compared to 2019. Even though I had a consistent weekly goal completion rate, I failed to reach several of my major goals. Most impacted when it comes to this blog and other goals was the drop in writing I did. I simply spent less time writing by a large margin and by extention wrote less words and published less articles. Interestingly, I had more meetings this last year than previous years, which may relate to a shift in my work and volunteering, but I also suspect it relates to a broader societal change as more people were available for video calls.</p>
<p>Let’s take a look at my year in tasks, projects and goals!</p>
<p>NOTE: This data was logged with <a href="http://www.markwk.com/task-tracking-with-todoist.html">Todoist</a> and <a href="http://www.markwk.com/goal-scoreing.html">my simple goal scoring system</a>. Data collection and visualization powered by <a href="http://www.markwk.com/qs-ledger-intro.html">QS Ledger</a>. This post is part of my <a href="http://www.markwk.com/category/year-in-data/"><em>2020 Year in Data</em> project</a>.</p>
<h3 id="total-completed-1708-tasks">Total Completed 1708 Tasks</h3>
<p><img src="/images/2021-resources/202101131221.25.png" alt="" /></p>
<p>Compared to <a href="http://www.markwk.com/2019-year-in-tasks-goals.html">2019</a> when I completed 1829 tasks, I completed 121 less tasks in 2020.</p>
<h3 id="weekly-goal-score">Weekly Goal Score</h3>
<p><img src="/images/2021-resources/202103151850.53.png" alt="" /></p>
<p>Noteable here is a drop in goal completion in week 37. All in all it was a fairly consistent goal completition rate</p>
<h3 id="monthly-tasks">Monthly Tasks</h3>
<p><img src="/images/2021-resources/202101131221.13.png" alt="" /></p>
<h3 id="breakdown-of-tasks-completed-by-project">Breakdown of Tasks Completed by Project</h3>
<p><img src="/images/2021-resources/202101131221.37.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131221.44.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131221.31.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131221.08.png" alt="" /></p>
<h3 id="creative-writing-by-word-count-from-aug-2019-to-jan-2021">Creative Writing by Word Count from Aug 2019 to Jan 2021</h3>
<p><img src="/images/2021-resources/202101131213.25.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131214.42.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131214.49.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131213.07.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131215.07.png" alt="" /></p>
<h3 id="what-tool-am-i-digitally-writing-and-taking-notes-with">What Tool am I digitally writing and taking notes with?</h3>
<p><img src="/images/2021-resources/202101131214.09.png" alt="" /></p>
<p>Winner: Typora and The Archive</p>
<p>NOTE: I suspect now that The Archive supports inline images that this usage rate may shift away from Typora going forward.</p>
<h3 id="how-i-track-my-tasks-and-goals">How I Track My Tasks and Goals</h3>
<p><strong>Todoist</strong>: For tasks and productivity management, I remain pretty married to Todoist as my primary task manager and task tracker. Todoist is a well-designed product that aligns with my <a href="http://www.markwk.com/gtd-with-todoist.html">Getting Things Done</a> productivity philosophy. As I wrote about in <a href="http://www.markwk.com/task-tracking-with-todoist.html">Tracking Your Tasks with Todoist</a>, Todoist has both an internal “gamified” task tracker and a solid integration with IFTTT. This enables me to create a Google Spreadsheet of my completed tasks and generate simple reports for my weekly and monthly tasks completed.</p>
<p><strong>DIY Weekly Goal Tracker and Scoring</strong>: I’ve been doing a <a href="http://www.markwk.com/data-driven-weekly-review.html">weekly review</a> and setting weekly goals for a number of years. A bit over a year ago I adapted a scoring system from Cal Newport for weekly goal “scoring”. The basic idea goes like this. Each week I set one or two major objectives and a few minor ones. At the end of each week, I give 1.5 points for each of the two major objectives. For the minor objectives, I take the percentage completed and multiply it by 3. I add these scores together to get my weekly score. The max score is technically 6 but I generally think about it as a score between 0 and 5. It’s simple and fun and keeps me accountable week to week.</p>
<h3 id="highlights">Highlights:</h3>
<ul>
<li>Total Tasks Completed: 1,708 (compared to 1829 tasks in 2019)</li>
<li>Daily Task Average: 5.6</li>
<li>Monthly Task Average: 142.3</li>
<li>291 tasks on Networking / Career (meaning I still did a lot of calls and meetings in 2020)</li>
<li>171 tasks on Productivity / Self-Tracking</li>
<li>161 tasks on Startup Boost</li>
<li>139 tasks on Studies</li>
<li>4.09 was my weekly goal score average, which means my goal-oriented outcomes were pretty aligned throughout the year.</li>
</ul>
<h3 id="conclusions-lessons-learned-and-looking-ahead">Conclusions: Lessons Learned and Looking Ahead</h3>
<p>While in many ways the lack of travel caused by COVID-19 meant I was probably more at home and “desk bound” than normal. So theoretically I could have been more productive and more task oriented than I was. Unfortunately I wasn’t as productive in terms of total tasks completed as I could have been.</p>
<p>All in all, my personal data analysis of my tasks revealed that I completed 120 or so less tasks in 2020 compared to 2019. I also didn’t hit many of my major goals for the year either.</p>
<p>I’ve long been an advocate of the goal-oriented life and productivty. But 2020 and COVID-19 had an impact on my attention, mindset and output. There was also a confluence of other factors in my life in 2020 that affects my work too, namely moving countries (from China to the US), higher financial and consulting demands now that I live in the US and new relationships.</p>
<p>There is no doubt that many, myself included, had to severely modify my goals I had set last year. No more travel and many of my epic goals needed to be paused as I took a break and figure out what mattered to me. That said, in view of my goal scores, I remained active and productive throughout the year. In fact, I’d argue that my goal score were steadier in 2020 compared to 2021, meaning I was generally more consistent in completed the goals I set.</p>
<p>In view of the on-going challenges around COVID-19 in my immediate surrounding, for 2021 I didn’t set <em>quite</em> as ambitious of goals as I could have if there was no pandemic. No plans for half or full marathons. No jet-setting to multiple companies either. Instead many of my health goals are around consistency and habits. Daily exercise. Regular and adeqate sleep. I am all about thinking about self-care and cultivated good mental health hygiene.</p>
<p>In spite of many limitations and challenges, I have set some ambitious targets for the year ahead. I want to write and publish more. Ideally at least a dozen or so blog posts and hopefully a book. My recent increase in creative word count and this blog post constiute a couple steps in that direction. The lockdown resulted in me learning the basics of music production last year. I plan to continue learning, making music and putting out a beat or song regularly on my <a href="https://soundcloud.com/djmystikos">soundcloud</a>. As a product designer and software engineer, I have plans to design and launch a couple products in the year ahead too.</p>
<p>All things considered I focus much less on a long-term vision and simply show up and do the “laps” necessary to make progress on things that matter to me.</p>
<p><br /></p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}My personal data analysis of my tasks reveals that I completed less tasks in 2020 compared to 2019. Even though I had a consistent weekly goal completion rate, I failed to reach several of my major goals. Most impacted when it comes to this blog and other goals was the drop in writing I did. I simply spent less time writing by a large margin and by extention wrote less words and published less articles. Interestingly, I had more meetings this last year than previous years, which may relate to a shift in my work and volunteering, but I also suspect it relates to a broader societal change as more people were available for video calls.A Year in Time: 20202021-03-15T00:00:00-07:002021-03-15T00:00:00-07:00http://www.markwk.com/year-in-time<p>There are 8760 hours in a year. In 2020, I spent, 1313.2 hours on my computer and another 1235 hours on my phone or tablet. Put another way, 2547 hours out of the total possible in 2020 was spent on a screen. This equates to about 29% of my life in 2020 that was screened. 1554 hours of that time was on projects, meaning consciously logged time on a project or specific task.</p>
<p>One additional screentime component that isn’t included here is my Kindle reading time. While it is also technically a screen, reading does quite feel comparable to these other devices. I read 52 books last year so at about 7 hours per book, that’s maybe 360 or so reading screentime hours.</p>
<p>Let’s look at and visualize my year in time data in more detail.</p>
<p>NOTE: This post is part of my <a href="http://www.markwk.com/category/year-in-data/">2020 Year in Data project</a> with data logged using RescueTime, Toggl and my iOS’s screentime feature and visualized using <a href="http://www.markwk.com/qs-ledger-intro.html">Quantified Self Ledger</a>.</p>
<h3 id="total-computer-time-13132-hours">Total Computer Time: 1313.2 hours</h3>
<p><img src="/generated/images/2021-resources/rescuetime_2020_heatmap-800-0c6f6c3e4.png" srcset="/generated/images/2021-resources/rescuetime_2020_heatmap-400-0c6f6c3e4.png 400w, /generated/images/2021-resources/rescuetime_2020_heatmap-600-0c6f6c3e4.png 600w, /generated/images/2021-resources/rescuetime_2020_heatmap-800-0c6f6c3e4.png 800w, /generated/images/2021-resources/rescuetime_2020_heatmap-825-0c6f6c3e4.png 825w" /></p>
<p><strong>500 hours less than 2019:</strong></p>
<p><img src="/generated/images/2021-resources/rescuetime_19_heatmap-800-a8667a63b.png" srcset="/generated/images/2021-resources/rescuetime_19_heatmap-400-a8667a63b.png 400w, /generated/images/2021-resources/rescuetime_19_heatmap-600-a8667a63b.png 600w, /generated/images/2021-resources/rescuetime_19_heatmap-800-a8667a63b.png 800w, /generated/images/2021-resources/rescuetime_19_heatmap-825-a8667a63b.png 825w" /></p>
<h3 id="top-computer-usage-month-july">Top Computer Usage Month: July</h3>
<p><img src="/images/2021-resources/202101131210.55.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131210.58.png" alt="" /></p>
<h3 id="top-apps--writing-and-dev-tools">Top Apps = Writing and Dev Tools</h3>
<p><img src="/images/2021-resources/202101131211.55.png" alt="" /></p>
<h3 id="top-computer-day-tuesday">Top Computer Day: Tuesday</h3>
<p><img src="/images/2021-resources/202101131209.54.png" alt="" /></p>
<h3 id="peak-computer-usage-around-10am-to-12-pm">Peak Computer Usage around 10am to 12 pm</h3>
<p><img src="/images/2021-resources/202101131211.05.png" alt="" /></p>
<h3 id="device-time-1235-hours-on-phone-and-tablet">Device Time: 1235 hours on Phone and Tablet</h3>
<p><img src="/images/2021-resources/202101131148.46.png" alt="" /></p>
<h3 id="the-big-blocks-of-my-project-time">The Big Blocks of My Project Time</h3>
<p><img src="/images/2021-resources/2020-01-01_to_2020-12-31_toggl_report.png" alt="" /></p>
<h3 id="total-project-time-1554-hours">Total Project Time: 1554 hours</h3>
<p><img src="/images/2021-resources/202101131157.38.png" alt="" /></p>
<h3 id="top-2-months-for-project-time-april-and-july">Top 2 Months for Project Time: April and July</h3>
<p><img src="/images/2021-resources/202101131156.41.png" alt="" /></p>
<h3 id="breakdown-of-workspaces">Breakdown of Workspaces:</h3>
<h5 id="development">Development:</h5>
<p><img src="/images/2021-resources/202101131157.54.png" alt="" /></p>
<h5 id="personal-projects-writing-and-studies">Personal Projects, Writing and Studies</h5>
<p><img src="/images/2021-resources/202101131157.52.png" alt="" /></p>
<h5 id="startup-projects">Startup Projects:</h5>
<p><img src="/images/2021-resources/202101131157.57.png" alt="" /></p>
<h3 id="breakdown-of-projects">Breakdown of Projects:</h3>
<p><img src="/images/2021-resources/202101131159.20.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131159.17.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131158.39.png" alt="" /></p>
<p><img src="/images/2021-resources/202101131200.18.png" alt="" /></p>
<h3 id="how-did-i-split-up-my-week-between-areas">How did I split up my week between areas?</h3>
<p><img src="/images/2021-resources/202101131156.46.png" alt="" /></p>
<h3 id="total-youtube-time-301-hours">Total YouTube Time: 301 hours</h3>
<p><img src="/images/2021-resources/202101131148.37.png" alt="" /></p>
<p>A couple of years ago I took a <a href="http://www.markwk.com/youtube-30-day-challenge.html">30-day break from YouTube</a>, which gave me better sense of both how much time I spent on YouTube and enabled me to use the video platform more towards learning rather than entertainment. I no longer use YouTube on the computer. Yet at nearly 50 minutes per day in 2020, I still think my usage is a bit high. Going forward, I want to try and limit my usage to around 4-6 hours per week while continuing to use it to learn and grow.</p>
<h3 id="highlights">Highlights:</h3>
<ul>
<li>1313.2 hours on computer (vs. 1840 hours in 2019)</li>
<li>702.8 hours on my phone (down from 895 hours in 2019)</li>
<li>532.7 hours on my tablet (down from 830 hours in 2019)</li>
<li>Meaning I had 2547 hours out of 8760 total possible in 2020 on a screen.</li>
<li>301 hours was on YouTube, most but not all of which was on my iPad.</li>
<li>1554 hours of project time (vs. 1716h in 2019), meaning about 29 hours per week I was consciously logged and engaged on a project or task.</li>
</ul>
<h3 id="how-i-tracked-my-time">How I Tracked My Time</h3>
<ul>
<li>For computer usage, I track it using RescueTime, a product I’ve used for several years and is integral for objectively knowing how long I spend on my computer and which tools I use most. For more on RescueTime, checkout <a href="http://www.markwk.com/data-processing-time-tracking.html">How to Create a Time Tracking Dashboard using RescueTime, IFTTT and Google Sheets</a>.</li>
<li>For project time tracking, I use Toggl, which is a free time tracker that works on web, mobile and Mac. <a href="http://www.markwk.com/2016/01/a-year-of-time-tracking-2015.html">Manual time tracking</a> does require some dedication but as a true metric of my input, it’s irreplaceable. Toggl has great reports, easy exports of raw logs and a nice API too.</li>
<li>For device tracking, each week I manually log my screentime from my iPhone and iPad into a spreadsheet. I’d love a better method, but since it only takes a minute and is part of my <a href="http://www.markwk.com/data-driven-weekly-review.html">weekly review</a>, it’s fine.</li>
</ul>
<h3 id="conclusion-lessons-learned-and-looking-forward">Conclusion: Lessons Learned and Looking Forward</h3>
<p>It’s crazy to think that 29% of my life in 2020 was “screened.” That said, compared to 2019 and 2018, my total computer time has been decreasing. I partially equate this to a bit more balance in my life. I exercise more regular, take regular breaks, and don’t obcessive about working all the time anymore. When I am at my computer, I avoid distractions and focus as much as possible on working on the tasks at hand and goals I have established.</p>
<p>When it comes to phone time, I am extremely gratiful for the mindspace I have recovered by using my phone less. I removed various apps and reconfigured my homescreen such that I have converted my phone into primarily a tool for photo taking, exercise tracking and communication, rather than entertainment and scroll. I’ll admit that TikTok videos are my biggest time suck but in view of how much laughter and joy it brings, I am less concerned about the occassional 30 minutes lost here and there.</p>
<p>Looking forward, I want to continue to strive at limiting my daily phone usage to less than an hour per day. Admittedly it’s ok to spend time using GPS, podcast listening and occassionally checking emails and messages, but in general my goal is to think of my phone as a productive enabler rather than a source of distraction.</p>
<p><br /></p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}There are 8760 hours in a year. In 2020, I spent, 1313.2 hours on my computer and another 1235 hours on my phone or tablet. Put another way, 2547 hours out of the total possible in 2020 was spent on a screen. This equates to about 29% of my life in 2020 that was screened. 1554 hours of that time was on projects, meaning consciously logged time on a project or specific task.Migrating to Jekyll, Again: How to Migrate Your Octopress Blog to Standard Jekyll2021-02-26T00:00:00-08:002021-02-26T00:00:00-08:00http://www.markwk.com/2021/02/jekyll-remigration<p>This blog is written in markdown, generated using Jekyll and hosted in plain old HTML. It’s all managed using Jekyll, a ruby-based static site generator, and it’s <em>roughly</em> the same setup I’ve had for nearly 7 years. I love it. But after being such a mainstay of my life as a blogger, it was time to migrate this site from Jekyll to…well…Jekyll.</p>
<p>After nearly 7 years hosting my blog on a flavor of Jekyll called Octopress, it was time to migrate my site from that setup on Jekyll to another one.</p>
<p>In this blog post I want to share a few challenges I faced migrating my blog from Octopress Jekyll to “Standard” Jekyll. First I want to share what I love about Jekyll and Why I had to migrate / update my site. Second, I explain a few issues I dealt with them in keeping certain features with more standard Jekyll. Specifically, we will look at how to retain liquid image tags, the archive page, dedicated category pages, a RSS feed, SEO, and a few other components. Hopefully, by the end of this post, if you are someone migrating from Octopress to “Pure” Jekyll, you’ll understand your migration and upgrading path. Let’s dig in!</p>
<h3 id="why-jekyll-and-a-plaintext-markdown-writing-life-rocks">Why Jekyll (and a Plaintext Markdown Writing Life) Rocks</h3>
<p>I’m a huge fan of the <a href="https://jekyllrb.com/">Jekyll Blogging Setup</a> setup for a number of reasons:</p>
<ul>
<li>First, since it’s hosted in HTML, there is no backend or database to manage.</li>
<li>Second, removing dynamic content and dedicated programming languages means it loads fast.</li>
<li>Third, Jekyll lets me write in markdown, a simple and clear plain text formatting syntax that only requires the basic text editor to manage. In fact, as I explain in one of my more popular posts, <a href="http://www.markwk.com/plain-text-life.html">The Plaintext Life</a>, I use markdown and plaintext files for nearly all of my work, including note-taking, writing and blogging.</li>
</ul>
<p>One of my first steps into the world of markdown and plaintext workflows was in 2013 when I elected to migrate my then Wordpress blog to Jekyll.</p>
<p>As I explained in <a href="http://www.markwk.com/2013/04/wordpress-migrations-to-octopress-blog.html">Migrating from WordPress to Octopress / Jekyll</a>, <a href="https://jekyllrb.com/">Jekyll</a> allows you to write files in markdown and then, using a few simple commands, generate a static, html site. The generated site can then be hosted on nearly any server, including Amazon S3 buckets as I do or using Github Pages.</p>
<p>When I switched this blog to Jekyll in 2013, I used one of the most popular and trending approaches to blogging on Jekyll, Octopress. Octopress offered a number of improvements to blogging in Jekyll and, to be honest, I liked the slick default theme it came with. I’ve really enjoyed Octopress and honestly wouldn’t mind continuing to use it today.</p>
<p>Fast forward several years and unfortunately, like many open source projects, Octopress is no longer being developed, and its support has wained. If I was retired or had additional open source dev time to spare, I’d love to contribute to its maintenance and on-going support. Unfortunately, while I’ve managed to keep the Octopress scripts working through several updates on Mac OS, my recent upgrade to Big Sur meant it was going to increasingly challenging using the version of Ruby supported by Octopress. So, in view of the lack of maintenance support, I decided it was time to “migrate off” of Octopress Jekyll and use a more “standard” Jekyll setup going forward.</p>
<p>Additionally, I wanted to keep as much as I could that I liked about Octopress, especially category pages and the main archive listing of blog posts. Like any web product I build and design for, a migration or upgrade was a good opportunity to sharpen the look, feel and usability of the product too. Even though the primary goal was migration and visual polish, it never hurts to improve usability too.</p>
<h3 id="how-to-migrate-your-octopress-blog-to-standard-jekyll">How to Migrate Your Octopress Blog to Standard Jekyll</h3>
<p>In order to migrate to Jekyll from Octopress, I recommend starting with a vanilla default version of Jekyll. So, go to <a href="https://jekyllrb.com/">jekyllrb.com</a>, download and setup Jekyll locally.</p>
<p>Once you have Jekyll running, you should be able to simply copy and past all of your pages, posts and assets into the new directory. There is a chance that your blog will work as is.</p>
<p>If not, don’t worry. It’s highly likely at this stage that you are going to hit the first of a series of hiccups in migrating. For example, if you use a few core features of Octopress (like image liquid tags syntax, category pages, or the blog archive), you’ll need to drop those features or find fixes. Let’s look at a few key fixes I made to get my site working:</p>
<h3 id="fixing-broken-images-enabling-liquid-like-image-tags">Fixing Broken Images: Enabling Liquid-like Image Tags</h3>
<p>Unfortunately, one biggest gotcha leaving Octopress are liquid image tags, which was basically a simple syntax for embedding image (<a href="http://octopress.org/docs/plugins/image-tag/">Ref</a>).</p>
<p>Unfortunately, though I didn’t know this at the time when I started using Octopress, this syntax is break with standard markdown which embeds images in this syntax: <code class="language-plaintext highlighter-rouge">![](path-to-image-file.png)</code></p>
<p>Since basically every single one of my posts has images embedded in this liquid tag syntax, I explored a few fixes. One option would be to parse and update all of my old posts to reflect standard markdown.</p>
<p>The other option I found and one I used involved adding the <a href="https://rbuchberger.github.io/jekyll_picture_tag/">jekyll_picture_tag plugin</a>, which not only provided the same syntax but also provided image versions optimized for screen size and mobile. The only bulk change I needed to make was to replace img with picture.</p>
<p>Though you may need to manually fix some posts, this plugin is a solid fix for migrating off of Octopress and keeping the same markup for images in your posts.</p>
<p>(NOTE: The only issue I still face here involves positional tags like left, right, and center which normally adds css markdown for positioning images around text.)</p>
<h3 id="wow-default-jekyll-looks-terrible-finding-a-starter-theme">Wow, Default Jekyll Looks Terrible: Finding a Starter Theme</h3>
<p>While structurally default Jekyll is great, it doesn’t really provide anything reasonably good looking out of the box. So one of the first things you’ll want to do is get a new theme or design.</p>
<p>One option is to create or adapt an HTML template. Since Jekyll is HTML centric, an HTML theme would be easy to adapt on a Jekyll blog.</p>
<p>Another option and the route I took was using a template on <a href="https://jekyllthemes.io/">jekyllthemes.io</a>. There are a lot of decent free options and nice paid options too. Many are adapted for specific use cases too. If you are looking to support open source and designers and developers (and to save your time), buy a theme you’ll love.</p>
<p>Admittedly as a designer, I could have designed and created a custom look and theme. But frankly, my goal on this site is to focus on writing and providing good content to my readers. So I bought a theme and did my tweaks on that. I just wanted a theme that looked great on all screen sizes and created an enjoyable and aesthetically pleasing reading experience. I didn’t want to spend days designing and developing when I could be writing and creating in other ways.</p>
<p>Following a few more minor tweaks here and there and an hour or two of manual blog edits, I went ahead and relaunched the site with my latest blog posts. Warts and all.</p>
<h3 id="setting-up-dedicated-category-pages">Setting Up Dedicated Category Pages</h3>
<p>Ok, now that we got our basic migration accomplished, we should have a functioning Jekyll blog and maybe even a decent theme too. Now what to do about some missing “Octopress” features on Jekyll? Can we fix those?</p>
<p>For me, one of the biggest features missing from switching to pure Jekyll for this blog was dedicated category pages. With Octopress, I really liked the idea of providing a single link for all of the writings I’ve done on a topic. I regularly share out and reference two of my most popular series via the category page, namely <a href="http://www.markwk.com/category/track-everything/">Tracking Everything</a> and the <a href="http://www.markwk.com/category/science-of-goals/">science of goals</a>. So I was sad when I migrated off of Octopress without dedicated category pages.</p>
<p>Unfortunately, while a lot of documentation on Jekyll claims that it is easy to create such pages, setting up category pages turned into the biggest time suck of this conversion. I ran into a number of issues with code I found on GitHub and fixes I found on several blog posts regarding generating category pages with Jekyll. Frankly I don’t know why I had so much trouble. For example, was it again a syntax issue or problems in the code I found.</p>
<p>Anyways, after several attempts, I finally found a solution using a couple snippets of code, which I aggregated and shared in a Github Gist <a href="https://gist.github.com/markwk/52d00f3f4e1ef1598fc2fb58f5b8ab7c">here</a>.</p>
<p>Here are the steps to make it happen:</p>
<ol>
<li>Create a file named GenerateCategoryPages.rb, add it to the _plugins directory and copy and paste the snippet below.</li>
<li>Create a file named category_index.html, add it to the _layouts directory and copy and paste the snippet below.</li>
<li>Regenerate your website using <code class="language-plaintext highlighter-rouge">jekyll serve</code> or <code class="language-plaintext highlighter-rouge">bundle exec jekyll serve</code>.</li>
</ol>
<p>For me getting my category pages working again was one of the highlights of this “challenge.”</p>
<h3 id="creating-a-archive-page">Creating a Archive Page</h3>
<p>While generating category pages required several attempts, fortunately generating a replacement archive page was easy. All you need to do is create a markdown file for a page can include the following code:</p>
<p><img src="/generated/images/2021-resources/archive-page-snippet-jekyll-800-c38849784.jpg" srcset="/generated/images/2021-resources/archive-page-snippet-jekyll-400-c38849784.jpg 400w, /generated/images/2021-resources/archive-page-snippet-jekyll-600-c38849784.jpg 600w, /generated/images/2021-resources/archive-page-snippet-jekyll-800-c38849784.jpg 800w, /generated/images/2021-resources/archive-page-snippet-jekyll-1000-c38849784.jpg 1000w" /></p>
<p>This code is a loop that will generate a list of past blog posts. In my case, I’m keeping it as just the date of publication and the title, but you could easily update this to include tags or categories or even feature images too.</p>
<h3 id="providing-an-rss-feed">Providing an RSS Feed</h3>
<p>I know that blog syndication via RSS or Atom feeds is what it used to be, but I still personally believe it’s a feature I wanted to maintain. This is probably a topic for separate rant someday, but I personally think RSS subscriber approach to blog and article consumption makes a lot of sense. It allows you to get updates from blogs or podcasts you follow and read and display them as you like.</p>
<p>Anyways, leaving aside my political opinions on decentralized content syndication, setting up an atom or rss is pretty simple with Jekyll. In my case I installed jekyll-feed plugin and tweaked the rss location so my long-time RSS subscriptions should continue to get updates.</p>
<h3 id="bonus-list-of-links-to-categories">BONUS: List of Links to Categories</h3>
<p>Here is a little <a href="https://gist.github.com/markwk/db151fc91eb8f08a2891fd0ac68a28a9">snippet</a> you can add to an html or markdown page to generate and list all of your categories as links.</p>
<h3 id="conclusions-and-meta-thoughts-on-blogging-going-forward">Conclusions and Meta Thoughts on Blogging Going Forward</h3>
<p>Hopefully this post provided some tips for dealing with common issues you might face migrating off of Octopress to more standard Jekyll. There are plenty of additional customizations and tweaks you might add to improve your blog. Jekyll is relatively easy to learn if you want to go deep on creating some pretty powerful content focused sites.</p>
<p>I personally think a markdown blog is a powerful enabler of creatives today. It’s easy to host it anywhere. You can even host it using Github pages for free. By using disqus, you get free comments too. While a Wordpress or Drupal site might be easier for blogging, they come with challenges too, especially around updates and databases. Jekyll removes many of these limitations and lets you focus on writing and publishing.</p>
<p>A couple years ago, I migrated off of Evernote and now use <a href="http://www.markwk.com/plain-text-life.html">plaintext for my note-taking</a> and my <a href="http://www.markwk.com/smart-notes.html">knowledge management system</a>. This means that nearly all of my research and note-taking as well as my final outputs and writings are all done in markdown files. This has enabled me to take on harder learning and writing challenges while also focusing on making writing and output of my writing as seamlessly as possible.</p>
<p>For me as a blogger, 2020 was a difficult year to write and publish. I had just moved to Los Angeles, California, roughly 4 months before COVID-19 resulted in a near continuous state of lockdown that is now going on nearly a year straight. My initial gratitude at <strong>not</strong> being in China when coronavirus struck has slowly pivoted into a degree of acceptance of just how poorly the United States as a government and society has dealt with this pandemic. It’s not been a great year anywhere and I’m glad to have had remote work and some positive impacts in my community. I’m still writing and creating, loving and living, especially around the intersection of tech and human betterment. I’m hopeful that things will change soon for the better, that we will all get vacinated, and that the United States and world will emerge stronger and more resilient.</p>
<p>Selfishly, after enjoying more time to be in one place, I’m also excited to travel again one day soon. In the meantime, I’m happy to be blogging and writing in the open again.</p>
<p>To borrow a phrase following my last blog migration 7 years ago that grew into a great sprout and period of blogging productivity: <strong>Readers beware</strong>.</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}This blog is written in markdown, generated using Jekyll and hosted in plain old HTML. It’s all managed using Jekyll, a ruby-based static site generator, and it’s roughly the same setup I’ve had for nearly 7 years. I love it. But after being such a mainstay of my life as a blogger, it was time to migrate this site from Jekyll to…well…Jekyll.Give Me My Data, Tell Me a Story: Introducing Quantified Self Ledger2021-02-11T00:00:00-08:002021-02-11T00:00:00-08:00http://www.markwk.com/qs-ledger-python-personal-data-analysis-tool<p>Several years ago I was exploring various self-tracking technologies and how I might integrate personal data into my own life. One of the first things I discovered was just how many ways you might <a href="http://www.markwk.com/category/track-everything/">track a life</a> using wearables, apps and other technologies. The second lesson I learned was just how hard it was to bring all of this data together and start engaging with it, even as a developer and fledgling data scientist.</p>
<p>While it is easier than ever to track a life, bringing all of your data together to visualize and compare it remains a real challenge. Personal data aggregation and data visualization was and has been the main motivation by my open source python data science project, <a href="https://github.com/markwk/qs_ledger">Quantified Self Ledger</a>.</p>
<p>When I started the project a couple of years ago, I was simply trying to share my own journey into improving my python and data science skills as well as make my code available to others too.</p>
<h3 id="a-personal-data-collector-and-data-visualization-for-a-quantified-self-and-a-data-driven-life">A Personal Data Collector and Data Visualization for a Quantified Self and a Data-Driven Life</h3>
<p>Initially QS Ledger was an assortment of python scripts to download and aggregate personal tracking data. I later added some additional notebooks to share examples of data analysis and data visualization. My primary goal was to use python to pull together a bunch of different streams of data and see if I could create something interesting with that data.</p>
<p>Much of my initial work was came out of a couple of <a href="https://github.com/markwk/python4selftrackers">data visualization and machine learning speeches</a> I gave at Python conferences and data science events, including <a href="https://rawgit.com/markwk/python4selftrackers/master/year-in-data-with-python/slides.html">A Year In Data with Python</a>. Having built a few tools in the quantified self and self-tracking space over the last couple years, including a <a href="http://www.podcasttracker.com/">podcast listening tracker web service</a> and <a href="http://www.photostats.io/">PhotoStats.io</a>, a private, mobile app for counting and tracking the photos you take on your phone, I was part of a huge initiative amongst developer to create various tracking services. Broadly speaking I’ve witnessed a rise in <a href="http://www.markwk.com/health-trackers.html">health tracking tools</a> as well as various methods to quantify our productivity, <a href="http://www.markwk.com/time-tracking-tools.html">time usage</a> and even <a href="http://www.markwk.com/track-writings.html">creativity</a>.</p>
<p>As I document in my <a href="http://www.markwk.com/quantified-self-mind-map.html">QS Mind Map</a>, if there is something you want to track, quantify or collect data on, then there is probably already a tool, app or service that can help you do it. But what to do if you want to look at, compare and use multiple personal data sources in one place?</p>
<p>Unfortunately the number of options for data aggregation and visualization are much more limited. Additionally, there are privacy concerns about many of these services too.</p>
<p>If data is more than something you want to collect, then QS Ledger might be a tool for you.</p>
<p>So what is QS Ledger?</p>
<!--more-->
<h3 id="introducing-quantified-self-ledger-an-open-source-python-data-science-project-for-the-data-enthusiast">Introducing Quantified Self Ledger: An Open Source Python Data Science Project for the Data Enthusiast</h3>
<p><a href="https://github.com/markwk/qs_ledger">QS Ledger</a> is open source python data science project that aggregates and visualizes your personal data.</p>
<p>The project has two primary goals:</p>
<ol>
<li><strong>download all of your personal data</strong> from various tracking services (see below for list of integration services) and store locally.</li>
<li>provide the starting point for <strong>personal data analysis, data visualization and a personal data dashboard</strong></li>
</ol>
<p>At present, the main objective is to provide working data downloaders and simple data analysis for each of the integrated services. What that means is that it can help you collect your data and offers examples on how to visualize your data.</p>
<p>If you just want to know more about the data you track, then QS Ledger can help you get your data and help you on the journey to exploring it.</p>
<p>If you are interested in creating year in review data infographics like I have in my <a href="http://www.markwk.com/category/year-in-data/">year in data series</a>, then QS Ledger can provide a great starting point.</p>
<p>If you are interested in Machine Learning and Artificial Intelligence on personal data, including health and wellness, then QS Ledger can help you bring together different data streams for predictive analytics and forecasting. I’ve even used this code to show correlations and make machine learning predictions about <a href="https://rawgit.com/markwk/python4selftrackers/master/slides/index.html">my own behavior over time</a> too.</p>
<p>Also, if you want to integrate data into a “Getting Things Done”-style <a href="http://www.markwk.com/data-driven-weekly-review.html">data-driven weekly review</a>, QS Ledger can help you create automated scripts for generating reports on your time, health or anything else you track.</p>
<h3 id="current-integrations">Current Integrations:</h3>
<p>QS Ledger currently integrations over a dozen data sources, including:</p>
<ul>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/apple_health">Apple Health</a>: fitness and health tracking, data analysis and dashboard from iPhone or Apple Watch (includes example of Elastic Search integration and Kibana Health Dashboard).</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/autosleep/autosleep_data_analysis.ipynb">AutoSleep</a>: iOS sleep tracking data analysis of sleep per night and rolling averages.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/fitbit">Fitbit</a>: fitness and health tracking and analysis of Steps, Sleep, and Heart Rate from a Fitbit wearable.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/goodreads">GoodReads</a>: book reading tracking and data analysis for GoodReads.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/google_calendar/">Google Calendar</a>: past events, meetings and times for Google Calendar.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/google_sheets/">Google Sheets</a>: get data from any Google Sheet which can be useful for pulling data from IFTTT integrations that add data.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/habitica/habitica_downloader.ipynb">Habitica</a>: habit and task tracking with Habitica’s gamified approach to task management.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/instapaper/instapaper_downloader.ipynb">Instapaper</a>: articles read and highlighted passages from Instapaper.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/kindle/kindle_clippings_parser.ipynb">Kindle Highlights</a>: Parser and Highlight Extract from Kindle clippings, along with a sample data analysis and tool to export highlights to separate markdown files.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/last_fm">Last.fm</a>: music tracking and analysis of music listening history from Last.fm.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/oura">Oura</a>: oura ring activity, sleep and wellness data.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/rescuetime">RescueTime</a>: track computer usage and analysis of computer activities and time with RescueTime.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/pocket/pocket_downloader.ipynb">Pocket</a>: articles read and read count from Pocket.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/strava">Strava</a>: activities downloader (runs, cycling, swimming, etc.) and analysis from Strava.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/todoist">Todoist</a>: task tracking and analysis of todo’s and tasks completed history from Todoist app.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/toggl">Toggl</a>: time tracking and analysis of manual timelog entries from Toggl.</li>
<li><a href="https://github.com/markwk/qs_ledger/tree/master/wordcounter">WordCounter</a>: extract wordcounter app history and visualize recent periods of word counts.</li>
</ul>
<p>Have an area you are tracking or tool you are using that isn’t include? Post a request in the <a href="https://github.com/markwk/qs_ledger/issues">issue queue</a> or add your own integration to the project.</p>
<h3 id="who-is-the-project-for-what-do-i-need-to-get-started">Who is the project for? What do I need to get started?</h3>
<p>While the project is generally for technically competent individuals, QS Ledger is intended as a beginner friendly data analysis and data science project. All you need is some basic knowledge of Python and a working installation of the Python Data Science stack. One good starting point is the <a href="https://www.anaconda.com/download/#macos">Anaconda Distribution</a>, which is cross-platform and will give you nearly all of the code you’ll need to start using QS Ledger.</p>
<p>In terms of key highlights:</p>
<ul>
<li>The code is written in Python 3.</li>
<li>Shared and distributed via Jupyter Notebooks.</li>
<li>Most services depend on Pandas and NumPy for data manipulation and Matplot and Seaborn for data analysis and visualization.</li>
</ul>
<h3 id="how-to-install-and-get-started">How to Install and Get Started</h3>
<p>Until we provide a working version for Google’s Collab or other online jupyter notebook setups, we recommend to get started by downloading and using the <a href="https://www.anaconda.com/download/">Anaconda Distribution</a>, which is free and open source. This will give you a local working version of Numpy, Pandas, Jupyter Notebook and other Python Data Science tools.</p>
<p>After installation, we recommend create and activating a virtual environment using <a href="https://www.geeksforgeeks.org/set-up-virtual-environment-for-python-using-anaconda/">Anaconda</a> or manually:</p>
<p><code class="language-plaintext highlighter-rouge">python3 -m venv ~/.virtualenvs/qs_ledger</code></p>
<p><code class="language-plaintext highlighter-rouge">source ~/.virtualenvs/qs_ledger/bin/activate</code></p>
<p>Then clone the current github repo:</p>
<p><code class="language-plaintext highlighter-rouge">git clone https://github.com/markwk/qs_ledger.git</code></p>
<p>Using your activate virtual environment, install dependencies:</p>
<p><code class="language-plaintext highlighter-rouge">pip install -r requirements.txt</code></p>
<p>Then navigate into your directory and launch an individual notebook or the full project with jupyter notebook or jupyter lab:</p>
<p><code class="language-plaintext highlighter-rouge">jupyter lab</code></p>
<p>After that, you can navigate to any number of data directories to continue your configuration and setup for the relevant service. For a deeper dive into one example with Apple Health, checkout my past post on <a href="http://www.markwk.com/data-analysis-for-apple-health.html">How to Export, Parse and Explore Your Apple Health Data With Python</a>.</p>
<h3 id="conclusion-tips-for-new-python-data-scientists">Conclusion: Tips for New Python Data Scientists</h3>
<p>On a surface, QS Ledger might look like a tool for experienced developers and technical folks, but speaking from my own journey to learning data science and machine learning, this is the kind of project and code for newbies too. In fact, one of my missions for this project is bring greater accessibility to personal data analysis and data science with Python. I highly recommend giving the project a try if you are new to python data science and you are looking for a way to work with new data sources.</p>
<p>Here are a couple tips for anyone getting started:</p>
<ul>
<li><strong>Completely new to python and data science?</strong> Check out <a href="https://www.amazon.com/Python-Data-Analysis-Wrangling-IPython/dp/1491957662/">Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython</a> by Wes McKinney as well as the tons of example jupyter notebooks you can find at Github.</li>
<li><strong>New to Personal Data Analysis? Start with a single data source</strong>: While it can be tempting both personally and as a developer to want to work with lots of different data points, I have found a ton of value focusing just one data source and personal data question at a time.</li>
<li><strong>Link your personal health goal with self-tracking and personal data analysis</strong>: One of the best ways I’ve found to reach my goals and change my behavior is through self-tracking. Even better when you pursue a goal is to link it with other projects you are working on. Considering getting more exercise or sleeping more, start tracking with Strava, Oura, Fitbit or Apple Health. Then using QS Ledger to create your own data visualizations.</li>
<li><strong>Focus on Project-Based Learning</strong>: While online courses and books can be a great start to your learning, there is no replacing working on projects. QS Ledger might be a great way to get started but you can only get so far with copy and paste code. Eventually you need “cut your teeth” by creating something on your own, ideally something no one else has done.</li>
<li><strong>Not sure what project to work?</strong> Reach out or post to the issue queue. I’m always looking for collaborators and happy to help anyone learning python data science too.</li>
<li><strong>Blog or Contribute to Open Source</strong>: As you learn and grow as a data scientist and technical person, be sure to write about in a blog or even better contribute to open source software. Nothing sets apart a candidate for a job better than open source code or a launched product.</li>
<li><strong>Stuck? Take a break and try again later</strong>. Good things come through working on hard projects and challenge. Also don’t be afraid to google or ask around for help.</li>
</ul>
<p>Good luck on your personal data journey. Please like and share QS Ledger with your friends. And if you are in need of a data-centric product person, contact me <a href="http://www.markwk.com/contact">here</a>.</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Several years ago I was exploring various self-tracking technologies and how I might integrate personal data into my own life. One of the first things I discovered was just how many ways you might track a life using wearables, apps and other technologies. The second lesson I learned was just how hard it was to bring all of this data together and start engaging with it, even as a developer and fledgling data scientist.Tracking Flow: Guide to Quantifying Momentary Mental States2020-03-17T00:00:00-07:002020-03-17T00:00:00-07:00http://www.markwk.com/tracking-flow<p><a href="http://www.markwk.com/science-of-flow.html">Flow</a> is a key mental state we experience when we become engrossed in an enjoyable challenge or activity. An athlete might feel flow in a competition or surmounting some obstacle, like climbing a mountain. A writer or painter might get into flow when working on a new creative project, like devising how to resolve a plot twist. A learner might feel flow trying to master a new topic or perform a new skill, like playing the guitar.</p>
<p>According to one of the chief researchers of flow, Mihaly Csikszentmihalyi, flow results from deploying our peak skills in near peak challenges. When the challenge is too easy, we are bored, and when it is too hard, we panic and freak out, but when the challenge is just the right balance of pulling at our skills, we can be triggered into a state of energized focus and peak performance called “the zone” or flow.</p>
<p>Flow matters for anyone pushing at their limits but is especially for creatives, since flow offers an optimal state where we are positioned to do and create our best work. A deeper dive into the <a href="http://www.markwk.com/science-of-flow.html">science of flow</a> has revealed a number of benefits to flow not the least of which is that it makes us feel happier.</p>
<p>I’ve personally been interested in flow for a number of years. I originally framed flow within the context of productivity, time management and <a href="http://www.markwk.com/category/getting-things-done/">getting things done</a> where it operated alongside other “deep work” practices, like eliminating distractions, scheduling uninterrupted blocks of work time, and <a href="http://www.markwk.com/category/weekly-review/">weekly reviews</a>. Even though I didn’t have a name for it, various changes in my life and time management had enabled me to create more flow in my life without realizing it. Flow came to dovetail nicely with <a href="http://www.markwk.com/category/time/">time tracking</a> and my effort to maintain a strong creative and professional output while decreasing my actual “work” time.</p>
<p>The short of it was that I had learned that, regardless of the project, flow was the optimal state for learning, creating, and working in my life. The more flow I had, the more often I was dealing with key challenges, projects, and goals and pushing myself to solve them. The more flow I had the more creative and productive I ultimately was.</p>
<p>Unfortunately, as a long-time <a href="http://www.markwk.com/category/track-everything/">self-tracker of everything</a>, a key question remained:</p>
<p><strong>How to track flow?</strong></p>
<p>For all of the excitement I’ve found around flow, tracking it is hard. Unlike sleep, heart rate or movement which I can measure with a wearable like an <a href="http://www.markwk.com/apple-watch-for-self-trackers.html">Apple Watch</a> or time spent on the computer or phone, flow can’t be tracked in a purely objective way yet.</p>
<p>Flow is largely subjective. It occurs in our brains and thus it is difficult to find an easy method of measuring it. Much like my previous experiments with <a href="http://www.markwk.com/category/mood-tracking/">mood tracking</a>, since flow is something we feel and experience, in order to track flow, we need to manually record and log it.</p>
<p>In this post, I want to look at how to track flow and other subjective mental states.</p>
<p>Personally, I believe a life lived with and for flow is worth living. Let’s get started looking at flow and how to track it!</p>
<p><br /></p>
<p><em>NOTE: This post is the second part of two part series on <a href="http://www.markwk.com/category/flow/">flow</a>. See Part 1 on the <a href="http://www.markwk.com/science-of-flow.html">science of flow</a>.</em></p>
<!--more-->
<h2 id="brief-what-is-flow">BRIEF: What is Flow?</h2>
<p><img src="/generated/images/2020-resources/science-of-flow-800-9c1564666.jpg" srcset="/generated/images/2020-resources/science-of-flow-400-9c1564666.jpg 400w, /generated/images/2020-resources/science-of-flow-600-9c1564666.jpg 600w, /generated/images/2020-resources/science-of-flow-800-9c1564666.jpg 800w, /generated/images/2020-resources/science-of-flow-1000-9c1564666.jpg 1000w" /></p>
<p>In the <a href="http://www.markwk.com/science-of-flow.html">Science of Flow: Nature’s Peak Performance Enhancer</a>, we took a deep dive into what is flow and its various features and markers.</p>
<p>Flow can be defined in two ways:</p>
<ol>
<li>On an operational level, flow occurs when situations challenge us as the appropriate limit of our skills. So, we might define flow as situations that properly challenge our skill level. This definition has helped to identify flow triggers and situations that induce flow, like mountain climbing, coding, painting, gymnastics, writing, etc.</li>
<li>On an experiential level, flow is something we feel. Flow is characterized subjectively as an optimal and even happy state of energized focus and hypervigilance. In this case it is sometimes referred to as being in the “zone.”</li>
</ol>
<p>Brain research into flow is still limited, but a number of studies provide some early clues on what flow does to our brains. For example, a neurochemical cocktail is activated in flow that enlivens our focus (norepinephrine), increases our mood, senses and pattern recognition (dopamine) and decreases fear and augments lateral thinking (anandamide).</p>
<p><img src="/generated/images/2020-resources/benefits-of-flow-400-a4c3f9e42.jpg" srcset="/generated/images/2020-resources/benefits-of-flow-400-a4c3f9e42.jpg 400w" /></p>
<p>From the perspective of brain scans of jazz improvisation, meditation and other flow-heavy tasks, flow changes our normal way of thinking and operating to what is now referred to as transcient hypofrontality. This refers to a series of remarkable changes in the frontal cortex of the brain.</p>
<p>Flow has a number of benefits. Flow has been known to</p>
<ul>
<li>Increase focus</li>
<li>Aid in peak performance</li>
<li>Energize</li>
<li>Augment our learning and skill-building</li>
<li>Make us happier</li>
<li>Improve creativity and problem solving</li>
</ul>
<p>When properly utilized, the mental state of flow and situations that induce flow help us be better humans. Flow ultimately brings us more focused energy. For learners, flow helps us learn more effectively. For creatives, flow can empower us to solve the stickiest of problems and approach obstacles in new ways. And for anyone struggling with distractions and getting things done, flow might a superpower you need to become more productive too!</p>
<p>So, now that we know what is flow, how might we track it?</p>
<h2 id="science-of-measuring-momentary-states">Science of Measuring Momentary States</h2>
<blockquote>
<p>Young or old, rich or poor, American, Asian, or European, people prefer to do almost anything more when they are with others than when they are alone…the presence of people helps to structure consciousness. But it is more than that: as social animals, we are attracted to other humans more than to most other stimuli in the environment. In their company we tend to feel more alive than when we are alone. (Hektner, 2007)</p>
</blockquote>
<p>Along with being one of the most important researchers on flow, Csikszentmihalyi introduced and developed a new methodology for studying human lives in the moment called “Experience Sampling.” Instead of retrospective diaries that ask you to recall what you did or felt in a previous day or week, his approach was to do check-in’s at the actual moment of that activity.</p>
<p>Csikszentmihalyi’s belief was that retrospective surveys introduce biases and skew results to the mean, meaning we under-report peaks and lows giving us merely the averages. While a professor at University of Chicago and later Claremont Graduate University, he and his team demonstrated this new method by using specialized timers that allowed for semi-random beeps to notify people to collect their activities, mental states and thoughts on their activities. This enabled them to collect how people are actually experiencing those activities in the context in which they happened.</p>
<p>In short, Experience Sampling Method (ESM), sometimes called ecological momentary assessment (EMA), aims at capturing life data as it happens. It removes cognitive biases in our retrospective memory and provides additional contextual data for unpacking our mental states and activities we are doing during them.</p>
<p>Csikszentmihalyi and fellow researchers have used EMS to study experience in a range of scenarios:</p>
<blockquote>
<p>(1) the psychology of adolescence, especially in educational settings;
(2) the experience of work, work stress, and work satisfaction;
(3) family dynamics and marital satisfaction;
(4) the experience of psychopathology (e.g., eating disorders, depression, substance abuse); (5) the experience of media, especially television viewing;
(6) cross-cultural comparisons of time use and quality of experience;
(7) gender differences in activities and in the quality of experience;
(8) solitude, friendship, and affiliation;
(9) the optimal experience of flow (Hektner, 2007)</p>
</blockquote>
<p>The team did important work on how we spend our time, including showing our moods and even how much time people spend watching TV. They showed that people get great pleasure and enjoyment watching TV, even if it doesn’t induce flow or heightened life satisfaction. They also showed that, in spite of a 40-hour work week, most only actually work around 27 hours per week. In and out of work context, most people spend 14-15 hour per week is socializing and eating.</p>
<p>In a series of studies on gender differences, experience sampling research also have revealed important gender biases particularly in the domestic chores done by women rather than men. For example, while men are increasingly involved in domestic activities, they still spend much less time than women on these chores. ESM has</p>
<p>Amongst many of the interesting insights they’ve found, two big discoveries stand out to me from ESM research.</p>
<p>First, by researching work and learning contexts as well as extreme sports and artists, researchers have found there is strong correlation between activities that induce flow and happiness. Basically by studying subjective states in the moment, they found that challenging and skill-demanding activities are most likely to make us happy. A life with more flow is a happier life.</p>
<blockquote>
<p>One of the earliest puzzles that ESM results presented had to do with people’s experiences on the job. On the one hand, workers—including service workers and those on the assembly line—reported some of their best experiences at work. Generally they felt more alert, skilled, and creative at work than they felt in free time at home. They also reported to be in the high-challenge, high skills “flow condition” when working on the job. At the same time, when paged at work they typically said they would have liked to do something else, whereas at home they had no desire to do something else, even when they felt passive and bored (Hektner, 2007)</p>
</blockquote>
<p>Second, while it was long-thought that our minds were generally task-oriented, ESM showed that much of our subjective mental state is in mind wandering. Mind wandering has been described as the shift in attention from task-related to task-unrelated thoughts. In fact, the default mode of our brain operation is now believed to be a stimulus-independent thought or mind wandering. Basically, much of what our brain and consciousness does is wander.</p>
<blockquote>
<p>The cumulative evidence from ESM studies suggests that almost the opposite is the case. When left to itself, the mind turns to bad thoughts, trivial plans, sad memories, and worries about the future. Entropy—disorder, confusion, decay—is the default option of consciousness. (Hektner, 2007)</p>
</blockquote>
<p>A deep dive into experience sampling method is beyond this post. If you want to dig deeper, I recommend checking out the wonderful book, <em>Experience Sampling Method</em> by Hektner, Schmidt, Csikszentmihalyi, which does a fantastic job summarizing the method and analysis. It shares many lessons learned by looking at people’s lives in the moment.</p>
<p>For our purposes around tracking flow, I’m going to focus on ESM Data Collection Forms (ESFs), which were developed and standardized across multiple studies to collect what we are doing and feeling in the moment. You can look at two sample forms <a href="/documents/20202/Sample-ESM-Data-Collection-Forms - Experience Sampling Method.pdf">here</a>.</p>
<h2 id="how-track-flow-or-any-other-subjective-mental-state">How Track Flow (or any other subjective mental state)</h2>
<h3 id="challenges-to-tracking-flow">Challenges to Tracking Flow</h3>
<p>There are a number of challenges to tracking flow. While I’d like to track flow objectively, externally and passively, several challenges and problems arise. Let’s look at a few of them.</p>
<p>As we saw in the last section, a good deal of research in flow involves neurology. In flow, certain regions of the brain are more active, and certain neurotransmitter levels are heightening. These markers are used to as indicators and possible neurological correlates into what happens in the brain during our experience of flow. EEG brainwave monitors might be one route towards a brain-based flow tracker. One day you might put on one of these devices, and it monitors various brainwave patterns, identifying your mental states during your work. This would enable us to ideally <a href="http://www.markwk.com/types-of-self-tracking.html">passively track it</a>. It might even use neurofeedback and other technology to enduce certain cognitive states. Unfortunately, as of right now, I don’t know of a single wearable you can use to track flow neurologically. Similarly there isn’t any way to track our neurochemistry either.</p>
<p>One alternative approach to looking into our brains and identifying flow there might be to look at our output and what we are doing. For example, we might track flow in some capacity by looking at how we spend our time and what tasks we completed. This is partially how Csikszentmihalyi did much of the early research into flow by tracking what people were doing. I currently track my time and tasks, so this could be a good way to at least track my deep work and maybe even could be considered a proxy metric for flow too. One requirement of flow is focused engagement and concentration. So blocks of time in a single application on a single task might be a good starting point for tracking the indicators of flow.</p>
<p>Thinking about my own work experience, I’m not always sure if my time allotment and the types of tasks I do would be a wholly accurate way to find flow. Some days I might be “in flow” and doing lots of hard tasks over long expanses of time, but other days I might not feel like I’m in a subjective state of flow but still accomplish a lot. I write, research, code, and advance on projects nearly daily now, regardless of whether the “muse” is there calling or not.</p>
<p>Could my time logs, tasks completed and calender reveal a pattern of flow or would it just show that I was occupied and busy? It is a fascinating question that definitely merits more data analysis and I have the data to do it, but for now I’m skeptical that this is enough to track flow. Instead, I think we need follow how much of the original research into flow was done and use subjective logging to track our flow.</p>
<h3 id="3-aspects-to-tracking-flow-subjectively">3 Aspects to Tracking Flow Subjectively</h3>
<p>Since flow is something we personally and subjectively experience and feel, it is difficult to track without a degree of manual tracking. To track flow, we have to note when we are in flow.</p>
<p>Similarly if we want to better understand what induces flow for us, including the time of day, context, which activities, etc., we need to record various contextual data points. Fortunately, scientists using experience sampling method (ESM) offer us a solid roadmap for tracking flow or any other subjective mental state.</p>
<p>Assuming you don’t want to use a diary method, then in order to track your subjective states, you need the following three pieces:</p>
<ul>
<li>Semi-random notification method: We need to be reminded at certain intervals to collect data.</li>
<li>Data collection method: We need a way to record our data on our activities, context and mental state.</li>
<li>Data collection questions: Questions that let us know about how we are feeling, thinking and doing a certain point in time.</li>
</ul>
<p>Traditionally, these have been done using a special watch that would send out notifications and a paper notebook to record your data. Researchers then transcribed and coded the responses for analysis. Nowadays computer and phone apps can bring this all together enabling you to get notified at regular intervals. There are even a few dedicated ESM research apps.</p>
<p>Before looking at the tech and apps, let’s look at the heart of this approach, our questions.</p>
<h3 id="core-of-esm-questions">Core of ESM Questions</h3>
<p>When I first started to track flow, I simply logged if I was in flow or not. I’d be pinged 5-7 times a day and state if I was feeling flow or not. While this one-question approach did work, it only told me about how often I was experiencing flow and didn’t provide much information on the relationship between flow and other factors, like the activity I was doing, location or even how energized I felt. In order to empower certain lifestyle design choices, I needed a more robust data collection method, which led me to ESM.</p>
<p><img src="/generated/images/2020-resources/esm-survey-sample-01-557-eeabeb599.png" srcset="/generated/images/2020-resources/esm-survey-sample-01-400-eeabeb599.png 400w, /generated/images/2020-resources/esm-survey-sample-01-557-eeabeb599.png 557w" /></p>
<p>Looking at sample forms, we see that ESM method provides questions to record the following in-moment data:</p>
<ul>
<li>What time is it?</li>
<li>Where are you?</li>
<li>What are you thinking about?</li>
<li>What is the main thing you are doing?</li>
<li>How do feel about that activity? Is it enjoyable? Are you skilled at it? Are you learning anything? Etc.</li>
<li>What mood are you in? Happy? Passive? Worried? Weak? Excited? Lonely? Angry? Etc.</li>
<li>Who are you with?</li>
</ul>
<p><img src="/generated/images/2020-resources/esm-survey-sample-02-542-6546f0038.png" srcset="/generated/images/2020-resources/esm-survey-sample-02-400-6546f0038.png 400w, /generated/images/2020-resources/esm-survey-sample-02-542-6546f0038.png 542w" /></p>
<p>While not all questions can be so clearly delinated, since many are about a relation between you and something, I find that at its core ESM is asking us to collect information on:</p>
<ul>
<li>Context (when, where and with who)</li>
<li>Main activity (what are you doing?)</li>
<li>Your relation to that activity in the moment and in general (i.e. enjoyable? skill-demanding?)</li>
<li>Your mental state at the moment (mood, motivation, energy, level)</li>
</ul>
<h3 id="my-subjective-check-in-questions">My Subjective Check-in Questions</h3>
<p>Based on ESM, in order to track the mental state of flow throughout a day as we enage in different activities, our flow tracker questions should capture the context, the activity, our mental state and our relation to the activity we are doing.</p>
<p>Here are my questions:</p>
<ul>
<li>What was the main thing you were doing?</li>
<li>Category of Main Activity</li>
<li>What else where you doing? (Optional)</li>
<li>Where are you?</li>
</ul>
<p>These questions I recommend for all flow trackers.</p>
<p>Additionally if you work in a shared office or do lots of collaborative work, then it would be good to include a question on who you are with. Since I work alone most of the time and my logging is focused on computer usage, I removed the questions related to who I was with.</p>
<p><img src="/generated/images/2020-resources/google-form-flow-tracker-01-668-6e64c4e78.png" srcset="/generated/images/2020-resources/google-form-flow-tracker-01-400-6e64c4e78.png 400w, /generated/images/2020-resources/google-form-flow-tracker-01-600-6e64c4e78.png 600w, /generated/images/2020-resources/google-form-flow-tracker-01-668-6e64c4e78.png 668w" /></p>
<p>Looking at my mental state in the moment, I use a linear or likert scale of 1 to 5 to measure my subjective state using these questions:</p>
<ul>
<li>Flow-y</li>
<li>Energy Level</li>
<li>Mood (Sad to Happy)</li>
<li>Motivation</li>
</ul>
<p>These questions aim to succiciently capture how I’m feeling with a focus on flow and energy level (which appear to be related in my experience). Unlike traditional ESM forms, I don’t have use nearly as many questions for my mood tracking. In view of my previous difficulties with <a href="http://www.markwk.com/category/mood-tracking/">tracking mood</a>, I’m ok with keeping mood tracking to merely valence (positive or negative mood).</p>
<p>About my main activity, I note:</p>
<ul>
<li>Enjoyable: Did you enjoy the activity you were doing?</li>
<li>Challenging: How challenging was the activity? (SKIP if Irrelevant)</li>
<li>Skill-Demanding: How skilled are you at doing that activity (SKIP if Irrelevant)?</li>
<li>Was the activity important to you?</li>
<li>Goal-Oriented: How important was activity to your future goals? i.e. does it impact your long-term objective?</li>
<li>Learning/Improving: Were you learning anything or getting better at something?</li>
</ul>
<p>I largely preserved the original ESM questions around my relation to what I’m doing. I believe there is value in capture details about the activity itself and my engagement to better understand how I might best work.</p>
<p>Overall, my flow and subjective state tracker includes 14 core questions and two optional bonus questions. While not in the original ESM form, I added a section for open comments as well as one for adverse effects, which is where I might note a headache, brain fog or some other sickness. Adverse effects is something used in supplement and drug studies. Neither question I answer regularly. In the end, I reduced and refined many of the questions in ESM sample forms, in order to shorten logging time. <strong>Now to actually log my subjective state takes me less than a minute</strong>.</p>
<p>I am not entirely sure of the value of all these questions, but I do feel they capture a range of information that is useful in understanding the activities I do work on and the mental state I bring to those activities. Along with my <a href="http://www.markwk.com/2008/02/how-much-is-my-time-worth-%25e2%2580%259cmy-time%25e2%2580%259d-the-economic-ultimate-valeur.html">time tracking</a>, calendar and <a href="http://www.markwk.com/task-tracking-with-todoist.html">tasks completed</a>, they provide good background data on understanding the effects of exercise, food, nootropics and beyond.</p>
<p>If my ultimate goal as a productive creative is enagement and flow, then I need to see how often I am actually in a state of energized and motivated deep work. This form of tracking helps me to do that.</p>
<h3 id="putting-it-all-together-a-quantified-self-approach-for-tracking-flow">Putting It All Together: A Quantified Self Approach for Tracking Flow</h3>
<p>As a long-time and dedicated self-tracker and quantified self enthusiast, I must admit that tracking my flow dovetailed from my previous efforts at <a href="http://www.markwk.com/category/mood-tracking/">mood tracking</a> and more recent experiments with biohacking and the effects of <a href="http://www.markwk.com/meditation-effect-on-cognition-experiment.html">meditation</a> and even nootropics. Essentially I realized that flow is a target mental state I should strive for in order to be more productive, to learn better, and to produce high quality creative outputs. Flow matter to me, so it made sense to try and track flow too.</p>
<p>As of right now, the best and arguablly only way to track flow is manually. Since flow like any mental state is largely subjective, it is an experience I myself have. Accordingly, tracking flow requires my conscious awareness and recognition of it. No one else can track it for me.</p>
<p>Over the years I’ve done a number of self-tracking experiments that require manual tracking, including <a href="#">mood tracking</a>, <a href="#">how often I poop</a> and <a href="#">Heart Rate Variability</a>. It is <a href="http://www.markwk.com/types-of-self-tracking.html">always preferrable to track passively</a>, but in this case it is unavoidable.</p>
<p>Fortunately there are a number of simple and reliable ways to log the data we need.</p>
<p>Our principal needs are:</p>
<ol>
<li>A way to get notified to do the data collection, and</li>
<li>A way to collect the data</li>
</ol>
<h3 id="diy-tracking-flow-from-a-computer-flexible-subjective-state-tracking-with-google-forms">DIY Tracking Flow from a Computer: Flexible Subjective State Tracking with Google Forms</h3>
<p>While mostly a generic survey tool, Google Forms is also good fit for tracking flow and other subjective states.</p>
<p>Forms is Google’s free tool for creating and collecting survey data. There are dozens of apps and website that will let you do something similar, but Google makes it easy and allows a seamless integration with their spreadsheet app, Google Sheets. I also use <a href="http://www.markwk.com/data-driven-weekly-review.html">Google Forms for Weekly Review</a> to record weekly information on my health, time, goals and more, which I then use to create comparative charts and graphs.</p>
<p>Like other survey tools, Google Forms lets you create a range of question types, like text, scale questions and multiple choice. When it comes to tracking flow, Google forms enable me to track key questions like activity I was doing. It also lets me create linear scale questions.</p>
<p>While not quite as clear as like-it or pure scale question type between 0 and 1, Google’s scale question allows me to track flow accordingly:</p>
<p><img src="/generated/images/2020-resources/google-form-flow-tracker-02-670-c5eb8d491.png" srcset="/generated/images/2020-resources/google-form-flow-tracker-02-400-c5eb8d491.png 400w, /generated/images/2020-resources/google-form-flow-tracker-02-600-c5eb8d491.png 600w, /generated/images/2020-resources/google-form-flow-tracker-02-670-c5eb8d491.png 670w" /></p>
<p>Excluding questions like where I am and activity category, all of my questions are linear scale type. Research indicates that a range of 5 to 7 is optimal, since beyond that number humans cannot make relevant distinctions.</p>
<p>Here is what my mood tracking valence question looks like:</p>
<p><img src="/generated/images/2020-resources/google-form-flow-tracker-03-661-6dfe00217.png" srcset="/generated/images/2020-resources/google-form-flow-tracker-03-400-6dfe00217.png 400w, /generated/images/2020-resources/google-form-flow-tracker-03-600-6dfe00217.png 600w, /generated/images/2020-resources/google-form-flow-tracker-03-661-6dfe00217.png 661w" /></p>
<p>While there some research indicating the benefits of randomly sorting order of questions, personally, I like having questions in a set order with a few different sections.</p>
<p>Now that we got our questions into a survey tool, the question is how do we get notified periodically to respond it?</p>
<h3 id="open-a-website-or-tool-with-semi-random-notification-methods">Open a Website or Tool with Semi-Random Notification Methods</h3>
<p>Once you have a survey tool for data collection on your flow and mental state, the next step is to setup a way to notify you to fill-in that survey.</p>
<p>Here are a few different approaches to do this:</p>
<ul>
<li><strong>[Mac / Windows] RescueTime to Trigger Survey Tool After Certain Amount of Computer Ueage</strong>: RescueTime is an <a href="http://www.markwk.com/time-tracking-tools.html">automatic time tracking</a>. I use it to trigger a notification and open certain web pages after tracking a certain amount of recorded time on your computer. For example, after an interval of one hour, two hours, 3h15, 5h and 7h, I receive a notification and my survey tool opens a brower tab to log what I’m doing and how I feel.</li>
<li>[Mac] Use Mac Automator to Automatically Open a URL in Your Browser: Automator is an app provided by Apple to create interesting recipes for improving how you work on the computer. <a href="https://www.makeuseof.com/tag/6-ways-use-mac-automator-browser/">Automator</a> can be configured to schedule urls to open in the browser at certain times of the day. For example, at 9, 10:30 12, etc.</li>
<li>[Mac/Windows] <a href="http://www.crontabs.org/">Crontabs</a> is tab scheduling extension for Chrome that lets you set intervals to open and close certain tabs. It can also be used to trigger times of day to open a URL of your choice.</li>
</ul>
<p>Additionally, you might use your calender or other classic reminder tools to find your own way to trigger these notifications to remember to log your flow. I personally use RescueTime to log my computer usage and then trigger a notification to fill-in the survey after a certain amount of time.</p>
<h3 id="tracking-flow-with-a-mobile-app-reporter-for-ios">Tracking Flow with a Mobile App: Reporter for iOS</h3>
<p><img src="/generated/images/2016-resources/reporter-app-data-750-827dd2e5a.png" srcset="/generated/images/2016-resources/reporter-app-data-400-827dd2e5a.png 400w, /generated/images/2016-resources/reporter-app-data-600-827dd2e5a.png 600w, /generated/images/2016-resources/reporter-app-data-750-827dd2e5a.png 750w" /></p>
<p>If you do not work predominately on a computer or wish to do more ubiquitous form of logging, then you’ll likely want to use a mobile app to do your flow tracking. Besides research-focused, which we will look at shortly, there are a few commerical options. My personal favorite is Reporter app.</p>
<p>While not specifically designed to track flow, <a href="http://reporter-app.com/">Reporter app for iOS</a> is a pretty unique life logging and quantified app that can be adapted for use in DIY experience sampling. Developed by <a href="http://feltron.com/">Nicolas Feltron</a>, a well-known artist and innovator in the data-driven life, Reporter triggers a notification several times a day and helps you log what you were doing, where you were (with GPS), who you are with, and how you feel. It can also be configured with any question you like too.</p>
<p>In one of my first flow tracking experiment involving nootropics and caffeine, I used Reporter with the following additional questions:</p>
<ul>
<li>Are you in flow? Yes/No</li>
<li>Energy Level (1-5)</li>
<li>Are you working? Yes/No</li>
</ul>
<p>This proved a powerful method to explore the possible correlation on how certain drugs modified my energy level and increased the likelihood of flow. It also revealed patterns on which activities were more likely to involve flow (like writing or coding).</p>
<p>Reporter is high configurable, both in types of questions and scheduling of notifications. The only downside is that can lead to you to use your phone more than you should. Like other <a href="https://github.com/woop/awesome-quantified-self/">technology and self-tracking tools</a> it can augment your experience sampling since self-tracking data allows you to record various aspects of your life, how you behave and see patterns.</p>
<h3 id="research-focused-esm-apps">Research-focused ESM Apps</h3>
<p>ESM has a number of research-focused apps and platforms. Even though these tools are not always the most user-friendly, research-designed survey tools that can be adapted for personal use. Here are a few examples:</p>
<ul>
<li><a href="https://pielsurvey.org/">PIEL Survey</a> - a free tool designed to gather survey data from people in their daily lives.</li>
<li><a href="https://app.expiwell.com/">EpiWell</a> - an integrated platform for building, scheduling and tracking surveys across mobile devices. Used by several universities, it offers a limited free version and a paid version for research studies.</li>
<li><a href="https://jeeves.cs.st-andrews.ac.uk/">Jeeves</a> from University of St Andrews - a free, android-only tool for designing survey tools for research purposes.</li>
</ul>
<p>Personally, I like PIELS Survey and have created a <a href="https://gist.github.com/markwk/fa24df31e6336e9d9375a14ca1aa2b46">sample control file focused on experience sampling and flow tracking here</a>. I still think it would be great to see a more generic tracking tool built along the lines of Reporter but with more data analysis and ways to run A/B experiments on yourself.</p>
<h2 id="conclusion-engaging-and-tracking-a-life-designed-for-flow">Conclusion: Engaging and Tracking a Life Designed for Flow</h2>
<p>Flow is a powerful mental state associated with situations where we are deploying our peak skills–mental and physical–in face of trying challenges. In flow, we feel energized, alive and focused, and, in most cases, it is when we perform at our best. As I’ve argued <a href="http://www.markwk.com/science-of-flow.html">elsewhere</a>, flow has a number of benefits from happiness- and energy-inducing to optimizing and enabling peak performance output in learning, sports, creativity, problem-solving and more. In short, in a world of distractions, flow, passionate engagement, and deep work matter.</p>
<p>Unfortunately for all of the benefits of flow in general in my own life, flow is hard to track. Since flow is largely subjective, it is an experience I myself have. We might see that a person is deep in their work or highly engaged in an activity, but it really is only the person experiencing that moment who can tell us if it was flow-y or not. Accordingly, as I explained in this post, tracking flow seems to require my conscious awareness and recognition of it. Like it or not, if we want to track flow we need to do it manually.</p>
<p>In this post, we looked the science of tracking momentary subjective states called experience sampling and showed how research-backed questions can be adapted to building your own flow tracker. Using Google Forms, Reporter or a number of other apps, you can setup semi-random notifications that trigger you to collect data on your activities and your subjective mental state during them.</p>
<p>By tracking your flow level, mood, energy and relationship to the activity you are doing, you can not only track how often you experience flow, but quantify the various associations that go along with flow. Basically, by tracking your flow, you can start to understand which activities induce flow and how different factors in your life (like sleep, exercise, caffeine or other drugs) might affect it. A flow tracker enables you to optimize a life designed for flow.</p>
<p>As of mid-March 2020, I have collected 300+ subjective state observations. I have been fairly consistent in my logging. Admittedly, when I started in Sept 2019 until late 2019, there were more weekly check-in’s (around 8-16 per week) than I have made in recent months (down to 5 to 10 per week). This has led to some skewing of results towards observations made in early periods. That said, even a simple breakdown reveals some interesting patterns:</p>
<p><img src="/generated/images/2020-resources/flow-tracking-results-01-566-e641a7790.png" srcset="/generated/images/2020-resources/flow-tracking-results-01-400-e641a7790.png 400w, /generated/images/2020-resources/flow-tracking-results-01-566-e641a7790.png 566w" /></p>
<p>While a complete data analysis goes beyond this post, a few simple observations are worth sharing now:</p>
<h4 id="my-flow-varies-from-week-to-week">My Flow Varies from Week to Week</h4>
<p><img src="/generated/images/2020-resources/flow-tracking-results-02-800-fed0385df.png" srcset="/generated/images/2020-resources/flow-tracking-results-02-400-fed0385df.png 400w, /generated/images/2020-resources/flow-tracking-results-02-600-fed0385df.png 600w, /generated/images/2020-resources/flow-tracking-results-02-800-fed0385df.png 800w, /generated/images/2020-resources/flow-tracking-results-02-899-fed0385df.png 899w" /></p>
<p>While I wish flow was a consistent state I could tap into in my life, a simple plot of my states as no flow, medium flow and high flow indicate that these vary from week to week. The most common occurence is mid-flow. Some weeks I might only get into high flow a few times and see a lot more no flow than others.</p>
<p>Since the last 6 months have witnessed several major life changes (including moving countries and new work), it’s not surprising my tracking and instances of flow have varied as well. It would be interested to compare the amount of flow I have and with number of <a href="http://www.markwk.com/task-tracking-with-todoist.html">tasks completed</a> or my <a href="http://www.markwk.com/goal-scoreing.html">weekly goal score</a>.</p>
<h4 id="certain-activities-like-writing-induce-or-trigger-flow-more-than-other-categories">Certain Activities (like Writing) Induce or Trigger Flow More than other categories</h4>
<p><img src="/generated/images/2020-resources/flow-tracking-results-03-800-35884f64e.png" srcset="/generated/images/2020-resources/flow-tracking-results-03-400-35884f64e.png 400w, /generated/images/2020-resources/flow-tracking-results-03-600-35884f64e.png 600w, /generated/images/2020-resources/flow-tracking-results-03-800-35884f64e.png 800w, /generated/images/2020-resources/flow-tracking-results-03-938-35884f64e.png 938w" /></p>
<p>The two categories with the most observations (Studies and Paid Work) had relatively low average flow scores, higher number of instances of no flow and a tendency towards mid-flow.</p>
<p>By contrast, writing and personal projects had the two highest average flow scores. They also both relatively few cases of no flow and a strong tendancy towards either mid-flow and high-flow.</p>
<p>In short, in my own life, while there are exceptions, I tend to not experiece much flow on client work. Instead, it is in writing where flow happens most regularly and most often for me. Even though there are fewer observations to validate the trend, I also had higher flow in data analysis and coding.</p>
<p>So, if I was looking for triggers for flow in my own life, it would be challenge personal projects and writing.</p>
<h4 id="my-energy-level-and-motivation-are-correlated-with-flow">My energy level and motivation are correlated with flow</h4>
<p><img src="/generated/images/2020-resources/flow-tracking-results-04-800-5d027a178.png" srcset="/generated/images/2020-resources/flow-tracking-results-04-400-5d027a178.png 400w, /generated/images/2020-resources/flow-tracking-results-04-600-5d027a178.png 600w, /generated/images/2020-resources/flow-tracking-results-04-800-5d027a178.png 800w, /generated/images/2020-resources/flow-tracking-results-04-1000-5d027a178.png 1000w" /></p>
<p>Using a correlation matrix, we can see that flow is more positively correlated with energy level and motivation rather than mood. Put another way, being motivated about a task and having a good baseline energy level tend to be more tied with being in flow. A positive or good mood does matter but doesn’t matter as much.</p>
<p>If I was looking for an actionable takeaway or two here, it would be that tasks and projects that motivate me matter for flow. This means that if a project doesn’t motivate me, it’s pretty unlikely I’ll get into flow.</p>
<p>Moreover, being energized is criticial if you want to get into flow too. So, it’s important to be eating right, exercising and getting enough sleep, so you are rested and come to your work energized. This correlation between energy and flow would also be an indicator why caffeine and nootropics (like modafinil) might help facilitate a state of flow too.</p>
<h4 id="my-data-dashboard-engaging-with-my-flow-tracking-data">My Data Dashboard: Engaging with My Flow Tracking Data</h4>
<p><img src="/generated/images/2020-resources/flow-tracking-dashboard-800-99c7a1e52.png" srcset="/generated/images/2020-resources/flow-tracking-dashboard-400-99c7a1e52.png 400w, /generated/images/2020-resources/flow-tracking-dashboard-600-99c7a1e52.png 600w, /generated/images/2020-resources/flow-tracking-dashboard-800-99c7a1e52.png 800w, /generated/images/2020-resources/flow-tracking-dashboard-910-99c7a1e52.png 910w" /></p>
<p>I’m a strong believer in the power of tracking and using data for understanding and improving various parts of our lives. From health and time management to productivity and creativity, I’ve seen how a <a href="http://www.markwk.com/category/track-everything/">data-driven life</a> has changed how I think and live my own life. Without data, it isn’t obvious what impacts what in our lives.</p>
<p>Unfortunately, I’ve also witnessed that merely having or collecting data often isn’t enough to bring value of that data into your life. The number of unused and abandoned wearables is a testement to the failure of tracking creating behavior change and human improvements. Personally, when it comes to <a href="http://www.markwk.com/health-trackers.html">health trackers</a> and other areas, it matters that we enage and leverage that data. For this reason, I not only track my flow but created a simple personal data dashboard in Google Data Studio to visualize that data week to week. By tracking my flow, my goals and other metrics, I can see patterns and take steps to priorized flow-inducing time management and activities in my daily life.</p>
<p>While tracking flow is hard and may not be a tracking activity that you can maintain long-term, tracking flow does offer clues to understanding differerent lifestyle changes and experiments I have run.</p>
<p>In the case of my own amoung flow, tracking data showed me that energy level and motivation are key aspects to reaching flow and that certain activities tend to trigger flow in me over others. Data also shown how instances of flow have ebbed in recent months, leading me to realize how difficult moving and setting up in a new city and home actually has been. This explains to some extent why I haven’t quite hit as many of the goals I had set for the first part of this year. I can’t expect to have the same degree of output if I’m not getting into flow with the same amount of regularity.</p>
<p>While I didn’t cover the topic in detail here, my flow tracking data is part of a broader effort at data-driven self-experimentation. For example, I used how often I experience flow to examine the effect of sleep and different psychoactive subsances. If I didn’t have a log of my flow, I wouldn’t have been able to objectively measure if and how these affected my flow. By looking at my flow logs in relation to how much sleep I was getting and nootropics I was taking, I’ve been able to see the benefits of both. Put another way, I’ve used flow data to valid that I need to get enough sleep to acheive flow and certain nootropics (modafinil, l-theanine, caffeine, etc.) can increase my own tendency to reach flow.</p>
<p>If peak performance and creative ouput are important to you, then you should design a life and environment to promote and maintain flow states. Flow is a key, maybe the key, mental state that any creative or learner should aim to understand and cultivate. By understanding and tracking flow, you can start to empower certain lifestyle design choices and optimize your own life for flow, creativity, learning, and more.</p>
<p><br /></p>
<p>Best of luck and happy tracking!</p>
<p><br /></p>
<h5 id="references">References:</h5>
<ul>
<li>Csikszentmihalyi, M., & Larson, R. (2014). Validity and reliability of the experience-sampling method. <em>Flow and the foundations of positive …</em>.</li>
<li>Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). <em>Experience Sampling Method</em>. SAGE.</li>
<li>Kotler, S. (2014). The Rise of Superman. Houghton Mifflin Harcourt.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Flow is a key mental state we experience when we become engrossed in an enjoyable challenge or activity. An athlete might feel flow in a competition or surmounting some obstacle, like climbing a mountain. A writer or painter might get into flow when working on a new creative project, like devising how to resolve a plot twist. A learner might feel flow trying to master a new topic or perform a new skill, like playing the guitar.Science of Flow: Nature’s Peak Performance Enhancer2020-03-05T00:00:00-08:002020-03-05T00:00:00-08:00http://www.markwk.com/science-of-flow<blockquote>
<p>Where is the lightning to lick you with its tongue? Where is the frenzy with which you should be inoculated? Behold, I teach you the overman: He is this lightning; he is this frenzy. – Friedrich Nietzsche (quoted in <em>Rise of Superman</em> by Steven Kotler)</p>
</blockquote>
<p>Flow is a mental state in which we are so engrossed in a challenging or enjoyable activity that nothing else matters.</p>
<p>In flow, we experience what might be called energized focus and often witness forms of peak mental and physical performance. Basketball players can’t miss a shot, a rapper’s words flow effortlessly or a conversation enfolds magically are all examples of flow or feeling “in the zone.”</p>
<p>Flow is an experience found in mountain climbers, writers, slackliners, artists, athletes, coders, sex, and many other professions. But it is also a mental state that can be found in everyday life activities too, like learning or an engaging discussion.</p>
<p>Flow is most consistently found when we do things that require our peak abilities. When the task is too easy, we feel lethargy and boredom. When the challenge is way too hard, we panic and stress out. But when the challenge is just hard enough, we are forced to engage our fullest capacities. This push to perform at the limits of our ability makes us feel alive and engaged. In flow, we often feel enjoyment, pleasure, and happiness.</p>
<p>There are evolutionary reasons why flow matters. Flow initiates a series of neurochemical changes that heighten our senses and increase our cognitive and motor skills. Flow makes us more capable of surviving when threatened, solving pressuring problems and dealing with challenges as an individual and in a group.</p>
<p>Not only is this same mental state primed and better for survival, creativity and peak performance; it’s also ideal for learning too. In fact, learning and creative activities like painting and writing are two examples where both masters and apprentices can experience a state of flow.</p>
<p>I’ve been interested in flow for some time, both from the side of improving how I optimize my learning and creativity as well as how one might track and quantify such momentary, subjective states. A few guiding books for me on flow have been <em>Flow: The Psychology of Optimal Experience</em> by Mihaly Csikszentmihalyi, <em>Rise of Superman</em> by Steven Kotler, and <em>Experience Sampling</em> by Joel M. Hektner. I’ve also found some of the elements of flow in the book by Cal Newport and concept called <em>Deep Work</em>.</p>
<p>Through these books and my own self-experimentation, I’ve come to realize just how powerful this mental state can be and I’ve come to organize my time, my life and my work around it. I also see flow as something a strong proxy for the benefits of biohacking, certain neuroenhancements and even aspects of transhumanism. Flow has served us for evolutionary history and will continue to serve as venture into space, combat diseases, and become more intertwined with technology and machine learning.</p>
<p>In this post, lets look at what is flow and why it matters for us as humans in the past, today and in future. Hopefully through an understand of flow, we can see why, as creatives, learners, peak performers and pursuers of excellence, we should look for flow as a key element in human flourishing.</p>
<p><br /></p>
<p><em>NOTE: This post is the first part of two part series on <a href="http://www.markwk.com/category/flow/">flow</a>. See Part 2 for a dive into <a href="http://www.markwk.com/tracking-flow.html">how to track flow</a>.</em></p>
<!--more-->
<h2 id="what-is-flow">What is Flow?</h2>
<p>In psychology and in our own observed lives, we experience and recognize a number of states of consciousness. The most obvious division is between wakefulness and sleep. But even there we start to notice in-between states too. For example, there is the weird half-awake, half-asleep state as we are falling asleep or waking up. This transition phase is called the hypnagogic and as its own unique subjective state and brain chemistry. During sleep, we also dream, and some people can also become lucid and can take control of their dreams. In short, consciousness isn’t simply an on-off switch between awake and asleep.</p>
<p>In fact, as Jeff Warren shows in his wonderful book, <em>Head Trip</em>, the “wheel of consciousness,” as he calls it, has at least 12 recognizable and scientifically demonstrated states. Noticeably for our purposes, Warren includes one separate state he labels “the zone.”</p>
<p>While going by a few different names, flow has been studied by various researchers for a long time, including by William James who might be called the father of modern psychology and more recently by contemporary psychologist Mihaly Csikszentmihalyi.</p>
<p>The roots of the concept of “flow” can be seen way back in William James’ idea of “mythical experience” (1902) and Maslow’s “peak performance.” Maslow studied exemplars of human performance and saw high achievers are intrinsically motivated and fall into intensively focused activity:</p>
<blockquote>
<p>“During a peak experience the individual experiences an expansion of self, a sense of unity, and meaningfulness in life. The experience lingers in one’s consciousness and gives a sense of purpose, integration, self-determination and empathy.”</p>
</blockquote>
<p>According to Csikszentmihalyi, a former professor at University of Chicago and one of the chief psychological researchers into flow, <strong>flow comes from deploying our peak skills in near peak challenges</strong>.</p>
<p>For him, flow emerges when we are in defiance against an obstacle at close to the limit of our abilities. These situations, often scary, sometimes life-threatening, require the best of us. Less challenging situations and not using our skills to their fullest will result in apathy and boredom or when an obstacle surpasses our skills, we become anxious and worried.</p>
<p>Peak challenges combined with deploying our peak mental and physical capacities, according to Csikszentmihalyi, result in flow.</p>
<h3 id="operational-definition-for-flow">Operational Definition for Flow</h3>
<h5 id="flow-y-situations-come-from-peak-challenges-at-the-limit-of-our-skills">Flow-y situations come from peak challenges at the limit of our skills</h5>
<p><img src="/generated/images/2020-resources/flow-definition-challenge-skill-600-194219662.png" srcset="/generated/images/2020-resources/flow-definition-challenge-skill-400-194219662.png 400w, /generated/images/2020-resources/flow-definition-challenge-skill-600-194219662.png 600w" /></p>
<p>Through a research methodology called Experience Sampling (which we will explore separately <a href="http://www.markwk.com/tracking-flow.html">here</a>), Csikszentmihalyi found among various populations a correlation between the type of activities people are engaged with and the mental states. Specifically, people were in flow and happier when they did activities that were challenging and required them to deploy their skills at their fullest.</p>
<p>This has lead to what might be called an operational definition of flow: <strong>Flow is what results from situations where people perceive the challenge as high (but not too hard) and posses and deploy their skills at their highest ability (but not too high to be beyond their skillset).</strong></p>
<p>Put another way:</p>
<blockquote>
<p>Inducing flow is about the balance between the level of skill and the size of the challenge at hand (Nakamura et al., 2009).</p>
</blockquote>
<p>Flow is a state where <strong>you feel in control over the tasks at hand</strong>, even when they are hard and challenging.</p>
<p>This rule of inducing flow or peak performance is sometimes calls the Yerkes-Dobson law where performance increases with physiological or mental arousal, but only up to a point. As stress increased, performance improves until you reach a certain limit and levels drop off or decline.</p>
<p>Shorthand for this would be to say flow comes from dealing with “just hard enough” challenges that progress over time with your improving skills.</p>
<h3 id="experiential-definition-of-flow">Experiential Definition of Flow</h3>
<h5 id="flow-is-an-optimal-state-of-consciousness-of-energized-focus-and-hyper-vigilance">Flow is an optimal state of consciousness of energized focus and hyper-vigilance.</h5>
<blockquote>
<p>Flow is an optimal state of consciousness, a peak state where we both feel our best and perform our best.
Steven Kotler, The Rise of Superman: Decoding the Science of Ultimate Human Performance</p>
</blockquote>
<p>For as much as the operational definition helps us know what induces flow, it’s worth returning to the roots of the concept of flow, peak performance, and classic psychology, since it also must be admitted that flow is an experiential and subjective state. We can and do experience flow. We can and do fall into “The Zone,” a state where time falls away, self-doubt is submerged and our actions, thoughts and bodily motions are smooth and unencumbered. We feel great too.</p>
<p>Flow is largely a subjective experience and a mental state of high concentration and hypervigilance. It is is characterized by profound mental clarity, heightened situational awareness, emotional detachment, and an automatic nature. We don’t think so much as act fluidly and thoughtlessly.</p>
<p>Flow can’t happen when we are distracted, since they take us away from our focus. So an experience of flow requires a relatively distraction-free environment.</p>
<p>Flow also can’t really happen in situations where you don’t feel motivated or care about the tasks at hand. So, to experience flow, you need to be motivated too.</p>
<p>For ages, artists, athletes and performers have described a sense of detachment in what they are doing and even believe they have an intuitive “voice” or “muse” guiding them. They believe they are merely the conduit of these inspired spirits. Essentially in flow we become less self-conscious and simply act.</p>
<blockquote>
<p>It was almost as if we were playing in slow motion. During those spells I could almost sense how the next play would develop and where the next shot would be taken. Even before the other team brought the ball in bounds, I could feel it so keenly that I’d want to shout to my teammates, “It’s coming there!”—except that I knew everything would change if I did. My premonitions would be consistently correct, and I always felt then that I not only knew all the Celtics by heart but also all the opposing players, and that they all knew me. —Bill Russell, 1960s basketball legend (quoted in Warren, J. 2009)</p>
</blockquote>
<p>There is some debate as to which athlete first called it the “Zone,” but what is flow seems to have gained its greatest foothold in athletics and how these sports performers overcome their self-doubt to perform at their best. The Bill Russell quote could be switched with any number of athletes during their most transcendent performance like Michael Jordan or Simone Biles.</p>
<p>While it might seem like flow is about pursuing peak performance, in fact research into flow has made several interesting discoveries. For example, people who experience flow tend to be happier, both in the moment itself and overall. Flow has been shown as a way to improve learning and accelerate the path to mastery.</p>
<p>Often characterized on an individual level, flow has also been shown to exist in groups, teams and communal situations, like a good conversation, strategizing and product design. Numerous organizations, companies, teams and militaries are all actively integrating flow as a crucial component to improved creativity, elevated performance and greater well-being individually and organizationally.</p>
<p>Flow isn’t a state of rest but a state of doing. <strong>Flow is the experience of being so involved in an activity that nothing else seems to matter. It’s a feeling of energized focus. Flow is an experiential state where you feel focused, confident, energized, motivated and happy in what you are doing.</strong></p>
<p>The short of it is that athletes perform best when their actions are automated, rather than rationally thought out. In fact, cognitively your thinking brain is too slow to be effective in these situations.</p>
<h3 id="the-brain-in-flow">The Brain in Flow</h3>
<p><img src="/generated/images/2020-resources/fmri-model-brain-in-flow-480-8bd4c30ee.png" srcset="/generated/images/2020-resources/fmri-model-brain-in-flow-400-8bd4c30ee.png 400w, /generated/images/2020-resources/fmri-model-brain-in-flow-480-8bd4c30ee.png 480w" /></p>
<blockquote>
<p>FIGURE 1: Three-dimensional surface projection of activations and deactivations associated with improvisation during the Jazz paradigm. (Limb, 2008)</p>
</blockquote>
<p>We now have two working definitions of flow, one that helps us understand what triggers flow (challenging situations that require heightened skills) and one that contextualizes the feeling of flow or “being in the zone” (focused, confident, energized, motivated and timelessly happy).</p>
<p>But what is happening in our brains during flow? What’s the brain like in flow?</p>
<p>Before we dive in, we need to note two things. First, research is still on-going about our brains and flow, and much of what we know about the brain is changing. Second, flow isn’t really a single state but a series or cycle of steps leading up to flow. As such, it is best to characterize multiple brain states before, during and after flow.</p>
<p>In Rise of Superman, Kotler describes the four stages of flow cycle as:</p>
<ol>
<li>The “struggle”: The situation facing you is hard. This is the pre-flow stage where you feel challenged and uncertain if you can even accomplish that thing or solve that problem. Examples include problem analytics, physical training, information gathering, intense prayer. It might even involve a good amount of time spent before the activity itself. While a separate topic, these are largely defined as “flow triggers.”</li>
<li>Release: To transition from struggle to flow, you must pass through a phase of taking your mind off the problem. This might mean going for a walk after a long study or problem-solving session.</li>
<li>Flow: This is the mental and peak performance stage we previous defined above.</li>
<li>Recovery: It is impossible to stay in a stage of flow indefinitely. This stage is the return to your normal, rational mind. You likely feel tired out and need time to rest and recover before going back into the challenge-flow cycle.</li>
</ol>
<p>During the experience of flow (much like creativity and learning), we see several remarkable and distinctive chemical and hormonal patterns in the brain. In the struggle phase, there is an uptick in adrenaline, which is part of flight-or-fight response and primes the body for flow experience. In the release phase, serotonin is released providing positive feeling and reinforcement towards repeating the experience or activity.</p>
<p>The three components of the”flow chain reaction” are:</p>
<ol>
<li><strong>Norepinephrine</strong>, which is responsible for data acquisition and leads to hypervigilance and focusing, increases.</li>
<li><strong>Dopamine</strong>, which is the often referred to as a feel-good hormone, increases leading to heightened ability to process data and increases our pattern recognition and the fluidity of certain motor skills.</li>
<li><strong>Anandamide</strong>, which is a psychoactive, endogenous cannabinoid (similarly found in marijuana), “elevates mood, relieves pain, dilates blood vessels and bronchial tubes (aiding respiration), and amplifies lateral thinking (our ability to link disparate ideas together).” Anandamide also decreases our ability to feel fear.</li>
</ol>
<p>As Kotler puts it:</p>
<blockquote>
<p>Consider the chain of events that takes us from pattern recognition through future prediction. Norepinephrine tightens focus (data acquisition); dopamine jacks pattern recognition (data processing); anandamide accelerates lateral thinking (widens the database searched by the pattern recognition system).</p>
</blockquote>
<p>Aided by this cocktail of neurochemicals, we see profound changes in our brains.</p>
<h3 id="transient-hypofrontality-the-signature-brain-state-of-flow">Transient hypofrontality: The Signature Brain State of Flow</h3>
<p><img src="/generated/images/2020-resources/fmri-model-brain-in-flow-2-670-68f5aa47d.png" srcset="/generated/images/2020-resources/fmri-model-brain-in-flow-2-400-68f5aa47d.png 400w, /generated/images/2020-resources/fmri-model-brain-in-flow-2-600-68f5aa47d.png 600w, /generated/images/2020-resources/fmri-model-brain-in-flow-2-670-68f5aa47d.png 670w" /></p>
<blockquote>
<p>FIGURE 2: Axial slice renderings of mean activations (red/yellow scale bar) and deactivations (blue/green scale bar) associated with improvisation during Scale and Jazz paradigms. In both paradigms, spontaneous improvisation was associated with widespread deactivation in prefrontal cortex…The scale bar shows t-score values and the sagittal section shows an anatomical representation of slice location; both scale bar and sagittal slice insets apply equally to Scale and Jazz data. (Limb, 2008)</p>
</blockquote>
<p>Neuroscientists, like Charles Limb (2008) who studied Jazz musicians and freestyle rappers, have found that the superior frontal cortex of brain deactivates when engrossed in certain activities. Similarly, in flow, we see a decrease in brain activity in prefrontal cortex. To put in more colloquial terms, in flow, the “thinking” part of our brains turns off.</p>
<p>Flow creates a state in the brain referred to as transient hypofrontality, which is believed to be the primarily signature of brain in flow. Sitting at the front of brain, unlike our lizard or ancestral brain, the frontal cortex is the “explicit” part of the brain associated with “rationality” and a “personal self.” It is a cognitively slow processing system, compared with the more automated and “self-less” parts of the brain like our cerebellum. It is partially believed to be where aspects of our consciousness and subjective sense of self arise.</p>
<p>During transient hypofrontality our sense of self or self-awareness disappears too. For example, in 2006, Israeli scientists discovered that when people “lose themselves in a task” like playing cards, climbing a mountain or having sex, a part of the brain called the superior frontal gyrus deactivates.</p>
<p>Time dilation is witnessed by those in this state. This is similar to brain scans done on Buddhist monks and Christian nuns by Newberg and D’Aquili who were studying the biology of spirituality and religious belief. During meditation or intense focus, you see orientation association area (OAA) or energy move elsewhere, such that a part of the brain is temporarily blinded. This led them to postulate that the experience of oneness or connection to God is result of this narrowing focus.</p>
<p>Similarly pleasurable and encompassing, flow has often been described by mountain climbers and extreme sport athletes as a oneness in their activity. They describe a transcendent connection with the mountain or an engrossing connection with their tool as a glassblower.</p>
<p>Put another way, on a neurological and subjective level, in flow we stop “thinking” and simply act; our “personal self” disappears and we enact. We are often simply the creative thing we do.</p>
<h2 id="conclusion-why-flow-matters">Conclusion: Why Flow Matters</h2>
<blockquote>
<p>“Don’t ask what the world needs. Ask what makes you come alive. Because what the world needs most is more people who have come alive.”<br />
Howard Thurman, civil rights leader (quoted in The Rise of Superman)</p>
</blockquote>
<p>Flow can be defined in two ways: First, on an operational level, we see flow occurs when situations challenge us as the appropriate limit of our skills. Second, on an experiential level, flow is something we subjectively feel as an optimal and even happy state of energized focus and hypervigilance.</p>
<p>While the brain science of flow is still early, a brain in flow reveals a neurochemical cocktail that enlivens our focus, increases our senses and pattern recognition and augments lateral thinking. This leads to what is now referred to as transient hypofrontality, meaning a series of changes associated in the frontal cortex of the brain found in jazz improvisation, meditation and flow.</p>
<p>Now that we know what flow is and the science behind it, it’s worth pausing to ask, why does flow even matter? Put another way, compared to other conceptions of the good life and happiness, why would one want to pursue a life oriented for and around flow?</p>
<p>In my mind, the principal benefits of flow are:</p>
<p><img src="/generated/images/2020-resources/benefits-of-flow-400-a4c3f9e42.jpg" srcset="/generated/images/2020-resources/benefits-of-flow-400-a4c3f9e42.jpg 400w" /></p>
<ul>
<li><strong>Attention-Focusing</strong>: The mental state of flow leads us to focus our attention on what’s important and what’s matters. The state of flow offers a break from a society increasingly distracted by social media, online videos and a host of activities that pull at and fragment our attention. In flow we feel focused and many of the other benefits emerge from simply being more present and aware.</li>
<li><strong>Peak Performance</strong>: Flow can lead to improved human performance physically and mentally. In his book, <em>The Rise of Superman</em>, Steven Kotler interviews numerous extreme sport athletes, many of who are at the peak of their sport and at the mortal limit of what’s feasible too. Each reveal how flow as a mental state enables them to do what they do from paragliding to big wave surfing. They also reveal how enlivened and happy flow makes them feel too. While many might also be labeled as “adrenaline junkies,” these athletes might be better called “flow junkies” since it isn’t just the risk and fear that drives them, but the heightened state of performance they touch on in while in those moments as well. The short of it is that anecdotal and some research evidence shows that flow makes us perform at a higher capacity than otherwise. So if you wish to push your creative, physical or even learning goals, flow is a key mental state to pursue too.</li>
<li><strong>Energizing</strong>: Flow requires and leads to an energized state. From tracking and experimenting with flow, I’ve come to realize a strong connection between flow and feeling energized. It is a two-way street. Flow does increase how energized and focused I feel. But in order to get into flow I need to feel energized and capable too. That means getting enough sleep, eating well, and avoiding too much negative stress allow me to be energized and get into flow.</li>
<li><strong>Superpowers our Learning and Skill-Building</strong>: The various neurochemical and subjective mental changes result in state of mind that learn faster and exponentially increase the rate of developing and honing our skills. Research has shown that flow accelerates the pace of learning. Learning can be a struggle, but a looking at <a href="http://www.markwk.com/science-of-learning.html">how we learn</a> reveals that the struggle before flow results in an optimal state in flow where we build new powerful connections and lasting memories.</li>
<li><strong>A Positive State of Stress in Flow = Happiness</strong>: I’ve personally struggled with <a href="http://www.markwk.com/on-knowing-happiness.html">defining what is happiness</a> and I am not convinced that happiness should be the guiding thread in my own life philosophy. Stress is often framed negatively, but there is an upside to certain kinds of stress and how we respond to it. Specifically, our response to stress offer a chance at a positive and healthy answer to challenges around us. Instead of closing down and panicking as we deal with challenging tasks at hand, the experience of flow can be thought of as a positive form of stress (sometimes called eustress). These positive responses help us to build resilience and a sense of accomplishment. Flow itself is highly enjoyable and, as flow researchers like Csikszentmihalyi have shown, they make us happier and increase life satisfaction.</li>
<li><strong>Less Self-Conscious, Increased Creativity and Problem-Solving</strong>: From looking at the brain in flow, we see a marked deactivation of our prefrontal cortex normally associated with higher rational functions, like planning and self-monitoring. Experientially flow translates to a state where we are “less in our head,” meaning we judge ourselves less critically and are less self-conscious. Somewhat ironically, these brain changes, peak performance benefits and improved rate of learning augment our capacity to be more creative. From studies on artists, athletes, musicians and more show that flow makes us not only feel more creative, but in fact, our creative outcomes are better too. Facing hard problems at the limit of our ability to deal with them isn’t really pleasurable per se, but this experience of being creative during flow are ultimately enjoyable, since situations that induce flow pull at our minds and bodies leading us to enact our own most creative responses possible. In short, they make us feel more alive and meaningful and subsequently more creative too.</li>
</ul>
<p>As a long-time self-tracker, quantified self enthusiast, and biohacker, I often look for trackable metrics to orient how I understand and improve my life. Among other things, I look at health data to help me better understand what makes me feel more energized, sleep better, and avoid lifestyle diseases, and I study my time tracking data, calendar and tasks completed to know where my time and attention goes and decide what goals I should do. It has even help me optimize certain tasks and goals for certain times of day.</p>
<p>In the past I’ve even turned to <a href="http://www.markwk.com/category/mood-tracking/">mood tracking</a> as a way to quantify how I feel in general and at certain points of the day. Mood tracking has failed me in the past since it’s a pretty limited slice of what matters in a life, namely how you feel. Obviously depression or a bad mood will severely impact your ability to be productive or creative, and we should strive to prevent these negative baseline mental states. But just “not being sad” doesn’t offer a framework for fulfilling life either, nor does the singular pursuit happiness or pleasure either.</p>
<p>Compared to happiness or mood, flow offers a more complex and enriching path towards human flourishing. Increasing flow in a life tends to be beneficial in so many areas. Flow can improve performance, enjoyment, energy level, creativity, happiness, learning, problem-solving and more. These benefits have lead me to design aspects of my life around flow and expand my tracking to include flow and related subjective metrics, like motivation and how energized I feel. I’ll leave the the topic of tracking flow for a future post. But we should never forget that flow doesn’t happen when the task at hand is easy. Tasks that are too easy bore us. Tasks that are too hard scare and intimidate us, leading us to stress out and panic.</p>
<p>To reach a mental state of flow we need to be challenged, often times at the very limit of what we are capable and where we will sometime fail. Put another, while a life of flow has many benefits and I personally believe it is a life worth living, it doesn’t and can’t occur unless we are pushing ourselves at the very limit of what is possible. Living in and for flow may be enjoyable but it isn’t easy.</p>
<p><br /></p>
<h5 id="references">References:</h5>
<ul>
<li>Csikszentmihalyi, M. (2008). Flow: The Psychology of Optimal Experience (Harper Perennial Modern Classics) (1 ed.). Harper Perennial Modern Classics.</li>
<li>Csikszentmihalyi, M. (1997). <em>Finding flow: The psychology of engagement with everyday life.</em> Basic Books.</li>
<li>Csikszentmihalyi, M. (1996). <em>Flow and the psychology of discovery and invention</em> (56). New York: Harper Collins.</li>
<li>Csikszentmihalyi, M., & Larson, R. (2014). Validity and reliability of the experience-sampling method. <em>Flow and the foundations of positive …</em>.</li>
<li>Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). <em>Experience Sampling Method</em>. SAGE.</li>
<li>James, W. (1902, 2014). <em>The Varieties of Religious Experience</em>. Masterlab.</li>
<li>Kotler, S. (2014). The Rise of Superman. Houghton Mifflin Harcourt.</li>
<li>Limb, C. J., & Braun, A. R. (2008). Neural substrates of spontaneous musical performance: An fMRI study of jazz improvisation. <em>PLoS one</em>, <em>3</em>(2).</li>
<li>Nakamura, J., & Csikszentmihalyi, M. (2009). Flow theory and research. <em>Handbook of positive psychology</em>, 195-206.</li>
<li>Newberg, A., & d’Aquili, E. G. (2008). <em>Why God won’t go away: Brain science and the biology of belief</em>. Ballantine Books.</li>
<li>Newberg, A. B., & d’Aquili, E. G. (2000). The neuropsychology of religious and spiritual experience. <em>Journal of Consciousness Studies</em>, <em>7</em>(11-12), 251-266.</li>
<li>Newberg, A., Alavi, A., Baime, M., Pourdehnad, M., Santanna, J., & d’Aquili, E. (2001). The measurement of regional cerebral blood flow during the complex cognitive task of meditation: a preliminary SPECT study. <em>Psychiatry Research: Neuroimaging</em>, <em>106</em>(2), 113-122.</li>
<li>Warren, J. (2009). <em>The Head Trip</em>. Vintage Canada.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Where is the lightning to lick you with its tongue? Where is the frenzy with which you should be inoculated? Behold, I teach you the overman: He is this lightning; he is this frenzy. – Friedrich Nietzsche (quoted in Rise of Superman by Steven Kotler)A Year in Writing and Creating: 20192020-02-06T00:00:00-08:002020-02-06T00:00:00-08:00http://www.markwk.com/year-in-writing-creating<p>This data was tracked and logged with a combination of tools (more on this below). The short of it is that <em>my number of creative words typed</em> was tracked with Wordcounter for Mac and <em>my project and smart notes</em> with my <a href="http://www.markwk.com/track-writings.html">open source, Git-based files writing tracker</a>. Data collection and visualization powered by <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>. This post is part of my <a href="http://www.markwk.com/category/year-in-data/"><em>2019 Year in Data</em> project</a>.</p>
<p>Below I’ll share how I track these metrics, walkthrough areas I want to improve in the tracking, data analysis and visualization and some lessons learned.</p>
<!--more-->
<h3 id="challenge-of-tracking-creativity">Challenge of Tracking Creativity</h3>
<p>Tracking your creativity is challenging. Even just tracking your creative output in words typed, blog posts published or code commits isn’t easy.</p>
<p>Oftentimes we try to relegate creativity to some subset of productivity and time management. As if understanding the time we put in can tell us something about the quality of creative output that comes from it. Admittedly organizational skills and an “organized mind” are just as important to an artist as they are to a management consultant. Steven King in his wonderful book, On Writing, talks a lot about how important it is to show and write and that there is a degree of craftsmanship to the so-called work of the muses. I agree that creativity does largely depend on the time we put into it. But besides tracking our time, can we track other aspects of our creativity?</p>
<p>For me, who has managed to track and visualize everything from my health, fitness and time to my reading and tasks completed, I’ll admit that I have struggled to find a way to capture what I create. Creativity is hard to summarize outside of the final outcomes.</p>
<p>The same chalenge hold trues for quantifying my learning and studies, As a life-long learner, my studies in a year may span history and science to technology and advanced algorithms and dozens of topics in-between. But how can we track what we learn, have learned or learning?</p>
<p>In the in the last year or so, as I’ve documented in <a href="http://www.markwk.com/plain-text-life.html">The Plain Text Life</a> and <a href="http://www.markwk.com/track-writings.html">Tracking Your Writings and Note-Taking</a>, I dramatically shifted how I write and create. The core creative principle stem from a book I read and reviewed called <a href="http://www.markwk.com/smart-notes.html">How To Take Smart Notes by Sönke Ahrens</a>. The key steps are:</p>
<ol>
<li>as I read books, articles and academic papers I take rough notes,</li>
<li>I then write out these notes in my own words into “smart notes” which include</li>
<li>references to the source of that idea and quotes.</li>
<li>Finally when it’s time to create something, I gather those smart notes and then transform into the final work, which in my case is a piece of writing, a presentation or a software product.</li>
</ol>
<p>This note-taking system has changed how I work and aided me in tackling more complicated topics. I no longer start with some outline that I expect to bend to my will. Instead, I read things that interest me, I collect passages and quotes, and I create smart notes.</p>
<p>This system gives me something quantifiable to my learning and creating: my smart notes. This is one number I’m following. Similarly I create clusters of ideas with tags and summary notes. In these clusters of smart notes and interconnections between notes, I believe kernals of creativity lie.</p>
<p>It’s stil too early to say if and how this system will work for me. But I can say that I’m now able to engage on “heady” topics I’ll study over the years ahead and I’ve noticed how my mind can emerge with new insights and ideas worth talking and writing about from the intersection of ideas and my notes.</p>
<h3 id="how-i-track-my-creativity-creative-output-and-studies">How I Track My Creativity, Creative Output and Studies</h3>
<p>The short of my writing and learning tracking is that I have plain text files where all of my knowledge notes, project notes and drafts are stored. This system of taking notes and writing enables me to now track most of my creative output too. I have a record of the first notes I take on a topic to the final product I produce, like an article, presentation or blog post. This setup also let’s me track much of creative process too!</p>
<p>Here are the tools I use to track my creative output and studies:</p>
<ul>
<li><strong>Wordcounter for Mac</strong>: This method lets me track how many my words I type. Built by the same team behind The Archive, one of my favorite <a href="http://www.markwk.com/writing-tools-2.html">note taking apps</a>, <a href="https://wordcounterapp.com/">Word Counter for Mac</a> is a configurable tool that keeps a record of how many words I type in each program per hour. To augment this tool, I created an <a href="https://www.packal.org/workflow/word-counter-app">Alfred Workflow integration</a> to make it easier to see your stats as well as wrote <a href="https://github.com/markwk/qs_ledger/tree/master/wordcounter">data analysis code in QS Ledger</a> for collecting and visualizing your stats over time. While my creative typing statistics doesn’t tell me anything about what I’m writing, it does like my time tracking reveal patterns of my creative routines.</li>
<li><strong>IFTTT Recipe for Blog Posts Published</strong>: I use a spreadsheet to track my blog publications. This is a bit of a hack using the automation tool IFTTT’s RSS feature. I created a simple <a href="https://ifttt.com/feed">IFTTT</a> recipe that detects new blog posts from my RSS feed and then adds a record into a Google Spreadsheet. After coping over legacy content, I have a dynamic way to see my publications over time.</li>
<li><strong>Git-based Writings Files Tracker</strong>: When I <a href="http://www.markwk.com/migrate-evernote-plaintext.html">migrated off of Evernote</a>, one of my favorite productivity tools, and onto a <a href="http://www.markwk.com/migrate-evernote-plaintext.html">plaintext file-based note-taking system</a>, one of the motivations was the potential to track my notes in more detail. The fruit of this labor is Writings Tracker, which I wrote about <a href="http://www.markwk.com/track-writings.html">here</a> and whose code is at <a href="https://github.com/markwk/writing-tracker">github.com/markwk/writing-tracker</a>. The script runs daily and looks at a directory of files to see if and what changed. The changes (like files modified and number of words added) get calculated and stored into a CSV, and a complete record of what changed is version controlled into a Git repo. This provides a complete record of any and all changes I made as well as summary statistics. I was only able to use the summary statistics in this current data analysis.</li>
<li><strong>What I Studied</strong>: While not high tech nor automated, in my monthly logging I keep a list of what I studied.</li>
<li><strong>How Many Smart Notes</strong>: I also write down monthly how many smart notes I took.</li>
</ul>
<p>While far from perfect, I believe I have established a good starting point for tracking my creative output (both in terms of my study notes and my blog posts and presentations). What I’m tracking, especially the git repo of notes, is in need of more data analysis work to glean more.</p>
<p>For example, I’d love to get a word cloud on my most used words or most written on topics. It would be fascinating to see which words, tags or topics appear throughout the year and clustered during certain time periods. Unfortunately I didn’t have time to do that type of analysis yet, but hopefully next year!</p>
<h3 id="creative-output-highlights">Creative Output Highlights:</h3>
<p>I’m fortunate to lead a creative life. As a public speaker, <a href="http://www.markwk.com/">writer</a>, and <a href="http://int3c.com/">product creator and developer</a>, my work can be divided into presentations, articles published and products built (or at least worked on).</p>
<p>Here are some highlights from my creative output in 2019:</p>
<h5 id="presentations">Presentations</h5>
<ul>
<li><strong>How to Be Creative: An Ideation and Innovation Workshop for Companies, Startups and Communities</strong>: I had the good fortune of sharing my thoughts on what is creativity, especially from the perspective of brain sciences, as well as doing a day-long hands-on workshop for finding and generating new ideas and then refining and strategizing them into products and solutions.</li>
<li><strong><a href="https://markwk.github.io/ts4health/slides/slides.html">Self Across Time: On Time Series Data Analysis</a></strong>: How does our health change over time? Can we account for it? Using Python’s Data Science and Machine Learning tools, I explored how we can apply techniques traditionally used in financial models to looking at health data, like our steps, sleep and heart rate. This topic was one of the hardest technical areas I worked through last year, and, while the results were somewhat unspectatular (due to lack of temporal patterns in my data set), I now have code snippets to apply this in future health data analysis. I’m especially excited to use it in my continous blood glucose study. Thanks to Pycon Malaysia for being a wonderful host!</li>
<li><a href="https://rawgit.com/markwk/python4selftrackers/master/year-in-data-with-python/slides.html#/title-slide"><strong>A Year in Data: Self-Tracking and Personal Data Analysis with Python</strong></a>: “I’ve used self-tracking and personal data analysis to change my life. And I believe technology and personal data can help others.” Building on previous presentations at various meetups and Pycons, I shortened and focused this talk as an intro to the <a href="http://www.markwk.com/quantified-self-mind-map.html">quantified self and self tracking</a>, <a href="https://github.com/markwk/qs_ledger">Python code for personal data analysis</a> and actionable steps to using tracking for changing your life. In short, this is my dive into the how to lead a data-driven life and how to use tracking tech and Python to do it!</li>
</ul>
<h5 id="blogs-published">Blogs Published</h5>
<p>I wrote <a href="http://www.markwk.com/">18 posts</a> last year, and it’s hard to pick my favorites. But a few stand out in my mind:</p>
<ul>
<li><a href="http://www.markwk.com/plain-text-life.html">The Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files</a>: My magnum opus for how to track notes.</li>
<li><a href="http://www.markwk.com/meditation-effect-on-cognition-experiment.html">Can Meditation Improve Your Attention? Self-Experiment into Mindfulness and Cognitive Testing</a>: I tested and tracked the effect of meditation on various psychological, cognitive tests of attention and reaction time.</li>
<li><a href="http://www.markwk.com/quantified-self-mind-map.html">Quantified Self and Self-Tracking Mind Map: Conceptualizing Tracking and Other Data-Driven Tech</a>: Conceptual map of all of the tracking tech out there.</li>
<li><a href="http://www.markwk.com/goal-scoreing.html">Scoring Your Weekly Goals</a>: Simple productivity “hack” to showcase weekly objectives and track if I acheive them.</li>
<li><a href="http://www.markwk.com/article-tracking-with-instapaper.html">Instapaper: Empowering How I Read Articles with Highlights and Tracking</a>: This post is for anyone who reads a lot of online articles but wants to keep better track of what you read.</li>
<li><a href="http://www.markwk.com/youtube-30-day-challenge.html">No YouTube: 30-Day Challenge</a>: YouTube is my biggest entertainment time suck on my phone and tablet. Here’s my attempt at giving it up.</li>
<li><a href="http://www.markwk.com/health-trackers.html">Data-Driven Health Trackers: An Actionable List</a>: A breakdown of technologies and techniques for tracking various dimensions of our health. A follow-up to <a href="http://www.markwk.com/tracking-health-or-habits.html">Are you tracking a health indicator or healthy habit?</a></li>
<li>
<p><a href="http://www.markwk.com/what-is-meditation.html">What am I meditating for? In Pursuit of A Definition of Meditation</a>: For such a popular behavior, we seem to struggle at how to define it historically and scientifically. One academic paper in particular helped me find a way to define this ancient yet modern practice.</p>
</li>
<li><a href="http://www.markwk.com/digitalize-paper-journals.html">Paper to Paperless: A Guide to Digitalizing Your Journals with a Scanner App</a>: I have a lot of analog notebooks. My attempt to bring paper journals into a digital notes system.</li>
<li><a href="http://www.markwk.com/most-digitalized-life-ever.html">Gordon Bell and The Epic Quest to Digitalize Everything</a>: So you want to track everything? Here’s an early innovator, err guinea pig, in pursuit of tracking a life down to the last pixel.</li>
<li><a href="http://www.markwk.com/multistage-goal-pursuits.html">Science of Goals: Goals as a Multi-Stage Pursuit</a>: Another dive into the science of goals, specifically from the perspective of action stages.</li>
<li><a href="http://www.markwk.com/goal-tracking-with-airtable.html">Goal Tracker for AirTable: A Flexible Tool for Goal Pursuit Tracking and Management</a>: My write-up on building a goal setting and tracker with AirTable.</li>
<li><a href="http://www.markwk.com/track-writings.html">Tracking Your Writings and Note-Taking</a>: Here’s one way to track all of your notes and writings using Git!</li>
</ul>
<h5 id="products-coding-and-things-i-built-or-updated">Products, Coding and Things I Built (or Updated)</h5>
<p>Besides several client projects where I built apps and websites, I worked on a number of my own <a href="http://www.markwk.com/products/">products</a>, including:</p>
<p><strong><a href="http://www.datadrivenyou.com">DataDrivenYou.com (Web)</a></strong>: Data-Driven You is the new web portal I built for content and community around self-tracking and data-driven goals. I’m working on a new podcast video series, a directory of tracking tech, and an online course. I’m also hoping to finally build a data dashboard tool for self-tracking and understanding if and how life changes affect us. Basically a data portal for running your own n=1 experiments!</p>
<p><a href="https://github.com/markwk/awesome-biomarkers"><strong>Awesome Biomarkers (Database) + Biomarker Tracker (Google Sheets</strong>)</a>: Two years ago I created and open sourced a list of blood biomarkers (See: <a href="http://www.markwk.com/what-are-biomarkers.html">What is a biomarker?</a>). While I still want to pursue building a <a href="http://www.biomarkertracker.com/">biomarker tracker app</a>, last year I used my open source biomarker list to create a way to track your biomakers with a Google Sheet. As a biohacker and health tracker, this tool makes it easy to track health metrics over time and clearly highlight at-risk areas.</p>
<p><strong><a href="https://github.com/markwk/qs_ledger">QS Ledger (Open Source, Python)</a></strong>: This is an open source project for aggregating various self-tracking data and visualizing it. Since starting and coding most of the heavy lifting two year ago, this last year I added a few more data sources and made it more automated using commandline shortcuts.</p>
<p><a href="https://github.com/markwk/attn-tracker"><strong>AttentiveAI App</strong></a>: At AT&T 5G Hackathon in LA, our team built an AI-enabled attention tracking app in React Native. The app detects faces and provides an attentive score. I created a custom model to detect and count hand raising using IBM Watson’s backend. Our original intention was to empower teachers with technology to understand and better management classroom behavior. The code is open source and we walked away with the prize for best usage of IBM Watson.</p>
<p><a href="http://manifestjournaling.com/"><strong>Manifest - an Adaptive Journaling App</strong></a> - This is my latest venture focused on extracting meaningful data and insights from journaling. Using natural language processing, we enable journals to record with their voice or text and then present them with a sentiment analysis and additional feedback. The original version called <a href="https://github.com/markwk/mindset_journaling_app">Mindset Journaling</a> was created in one weekend by me at hackathon with AI LA and Cedar-Sinai Accelerator. Following customer interviews and market research, we are currently developing our first version and expect to launch in early 2020!</p>
<h3 id="what-i-learned-from-the-data">What I Learned from the Data:</h3>
<p><img src="/generated/images/2020-resources/writing-tools-tracking-numbers-800-3d23b6a9e.png" srcset="/generated/images/2020-resources/writing-tools-tracking-numbers-400-3d23b6a9e.png 400w, /generated/images/2020-resources/writing-tools-tracking-numbers-600-3d23b6a9e.png 600w, /generated/images/2020-resources/writing-tools-tracking-numbers-800-3d23b6a9e.png 800w, /generated/images/2020-resources/writing-tools-tracking-numbers-1000-3d23b6a9e.png 1000w" /></p>
<p>By far my favorite two writing apps on Mac are Typora and The Archive. As I share in <a href="http://www.markwk.com/writing-tools-2.html">How I Write: My Favorite Tools and Apps for Writing</a>, both are markdown writing tools. This chart (along with the others) show a marked spike in writing in summer and decrease into the fall and winter.</p>
<p>Unlike 2018 when I published 25 posts, I only published 18 blog posts in www.markwk.com in 2019. This was below my intended goal.</p>
<p>Looking at my writings data, two things stand out:</p>
<ul>
<li><strong>More Creative Typed Words</strong>: My total typed creative word count was 491,927 in 2019 compared to 650,939. In short, I typed more creative words in 2019 than 2018!</li>
<li><strong>Less “Writing” Project Time</strong>: According to my <a href="http://www.markwk.com/2019-year-in-time.html">time tracking</a>, I logged 70 hours less time writing in 2019 (300.3 hours) compared to 2018 (370.9 hours). Compared to the previous year’s totals, this adds up to less than an hour a day.</li>
</ul>
<p>If I was looking for one reason why I published less in 2019 (besides the major life change in Q4!), one culprit would be that I spent less time writing!</p>
<p>As I explain more below, this is likely do to the higher amount of time I spent on going deep and researching certain topics I have yet to write or publish on. For example, I did a month of studies on Tea, Qingchengshan and Daoism, but I haven’t published anything on the topic. Similarly I only did a presentation on the topic of creativity but didn’t publish a blog post on the topic yet.</p>
<p>Unfortunately my current tracking doesn’t tell me the length of blog posts compared year to year, so I can’t be sure if I actually published significantly less words or not.</p>
<p>Looking at my smart notes count, which is how I quantify how I learn inside my study method, we see another sign behind why my creative output dropped somewhat in 2019. Along with less blog posts I was taking less smart notes too. For me this decrease in digital note-taking is the strongest indicator for why my writings output dropped too. Basically, if I want to have complete ideas to publish on, I need to have already written and created good smart notes to start from, since most of the time writing is about transformation (rather than ex-nihilio inspirational moment).</p>
<h3 id="what-i-learned-this-year">What I Learned This Year:</h3>
<p>As part of my <a href="http://www.markwk.com/data-driven-weekly-review.html">monthly reviews</a> and a general log I keep on what I’m learning, I record the topics I study each month. This isn’t a very sophisticated form of tracking (at least compared to some of the data here or in my <a href="http://www.markwk.com/2019-year-in-reading.html">year in reading</a>), but it does give me a month-by-month breakdown of what I learned in the past year:</p>
<ul>
<li>January: Book: War on Art, Habits, Marine Biology (esp Cephalopods), Flow</li>
<li>February: Javascript, Flow, Tea, Self-Tracking, Qingchengshan</li>
<li>March: Goals: Model of Action Phases, Implementation Intentions, Daoism, China History, Qingchengshan, Quantified Self Background, Feminism</li>
<li>April: Feminism, Masculinity, COURSE: Flask API, Economics</li>
<li>May: Economics, New Orleans History, Alexander Hamilton</li>
<li>June: Bitcoin, Jet Lag, Brain and Neurology Studies (ex. Nootropics, Cognitive Testing, Modafinil, L-Theanine)</li>
<li>July: Health Studies, Brain and Neurology Studies (inc. Neurotransmitters and Cognitive Testing), Data Science Studies (Data Viz, Time Data Visualization, Crossover and N=1 Experimental Design, Statistics), Health Numbers, Experience Sampling, Creativity</li>
<li>August: Health Studies, Time Series Data Visualization and Analysis, GRE Studies (especially Math Review), Experience Sampling, Time Series Modeling</li>
<li>September: Skin Fluorescence, GraphQL, Behavioral Economics</li>
<li>October: Classroom Management, Attention Tracking, React Native, Information Economics (Information Assymetry and Signalling), Product Management, Caffeine, Mental Health (like Flourishing and Positive Psychology)</li>
<li>November: GRE Studies (especially Math Review), React Native, Positive Psychology and Flourishing</li>
<li>December: Javascript, React.js and Gatsby, UX Strategy, Phenomenology via World Beyond Your Head, AWS Amplify for Mobile Backends, Natural Language Processing</li>
</ul>
<h3 id="looking-ahead">Looking Ahead:</h3>
<p>My two writing goals in 2019 were 1.) to write longer, well-researched articles and 2.) to publish 24 blog posts.</p>
<p>I definitely completed several long and deeply researched articles this year, like <a href="http://www.markwk.com/what-is-meditation.html">What is Meditation?</a>, <a href="http://www.markwk.com/multistage-goal-pursuits.html">Goals as a Multi-Stage Pursuit</a> and <a href="http://www.markwk.com/most-digitalized-life-ever.html">Gordon Bell, the Most Digitalized Life Ever</a>. I also created a few posts that required a good deal of data analysis (like <a href="http://www.markwk.com/meditation-effect-on-cognition-experiment.html">Does Meditation Affect My Cognition?</a>), coding (like <a href="http://www.markwk.com/track-writings.html">Writings Tracker</a>) or other product work (like <a href="http://www.markwk.com/goal-tracking-with-airtable.html">Goal Tracker for AirTable</a>.</p>
<p>Like my current <a href="http://www.markwk.com/category/year-in-data/">series on a year in data</a>, I find these types of long, deep dives richly rewardly and fulfilling. But the effort is not insignificant. Unlike some past posts that merely introduced how to track something, I’m increasingly interested in the science behind whatever we are trying to quantify as well as what we can learn from these numbers. Combining the science of behavior with technologies that help us track and modify it is central to much of my product and social work today.</p>
<p>While on the surface these two writing goals seem complementary, I now realize that these goals (write longer vs. publish a certain quantify) are somewhat discordant or in conflict. It’s hard to research and write in depth and still expect to have the same numerical output. Push come to shove, I’m glad I went with depth over some arbitrary number.</p>
<p>That said, the numbers don’t lie. I didn’t spend as much time on my writing as I did in 2018. I’d like to get this writing time number closer to where I was, meaning about 1 or 2 hours a day dedicated to writing. Similarly, while I had a great start and solid summer creating “smart notes,” this lagged later in the year. I still <a href="http://www.markwk.com/2019-year-in-reading.html">read a ton</a> but I need to get back in the habit of processing what I read into notes on key takeways and insights.</p>
<p>Overall, I’m super excited about the year ahead. I have a solid list of things I want to write, create and learning. Now that I’m in the US more long-term, I’m especially keen to present more and focus more on creativing data-driven, tracking tools in line with my big vision. Additionally, I can’t wait to see what I can glean from a deep data dive into tracking my creative output more too!</p>
<p><br /></p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}This data was tracked and logged with a combination of tools (more on this below). The short of it is that my number of creative words typed was tracked with Wordcounter for Mac and my project and smart notes with my open source, Git-based files writing tracker. Data collection and visualization powered by QS Ledger. This post is part of my 2019 Year in Data project.A Year in Reading: 20192020-02-05T00:00:00-08:002020-02-05T00:00:00-08:00http://www.markwk.com/year-in-reading<p>This data was logged with <a href="http://www.markwk.com/2016/11/book-reading-tracking.html">Goodreads</a>, Kindle and <a href="http://www.markwk.com/article-tracking-with-instapaper.html">Instapaper</a>. Data collection and visualization powered by <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>. This post is part of my <a href="http://www.markwk.com/category/year-in-data/"><em>2019 Year in Data</em> project</a>.</p>
<h5 id="highlights">Highlights:</h5>
<p>Another good year of reading with lots of learnings coming from studies on economics, flow, feminism. I read a lot more nonfiction than fiction than in previous years. If I had to recommend a single book from 2019, it would be <em>23 Things They Don’t Tell You About Capitalism</em>. I didn’t agree with many points in the book, but I did find new evidence, arguments and perspectives about our economic world, especially economic development models.</p>
<h5 id="how-i-tracked-it">How I Tracked It:</h5>
<p>I <a href="http://www.markwk.com/2016/11/book-reading-tracking.html">track my book reading</a> with a combination of tools. Goodreads helps me track books read (and discovery new books based on past readings). My Kindle lets me collect and track book highlights. For articles I use <a href="http://www.markwk.com/article-tracking-with-instapaper.html">Instapaper for tracking my article reading and article passage highlights</a>. While not visualized here, I use Bookends to track my academic article references and use Skim app to extract my PDF highlights. I also wrote a script that tracks PDF changes, which I should be able to use to do a comprehenive analysis of PDF readings one day.</p>
<h5 id="what-i-learned">What I Learned:</h5>
<p>From the data and visualizations, it appears that I read more in the early months of the year. My books read, highlighted passages and articles read all decreased in Q4, which corresponds to a number of life changes (witnessed in my <a href="http://www.markwk.com/2020-year-in-time.html">time</a> and <a href="http://www.markwk.com/2020-year-in-tasks-goals.html">tasks</a> data too) as wel as some non-book related studies I was doing.</p>
<p>Looking at the word cloud, it appears that, like many other people out there, Trump was a popular topic in my article reading. It’s clear I also was reading a lot about China, health, the brain and tea too.</p>
<h3 id="my-top-books-from-2019">My Top Books from 2019</h3>
<p>It’s difficult to pick my favorite books since there is a lot of memory bias depending on when I report this. You can checkout my Goodreads Reviews and Rating <a href="https://www.goodreads.com/user_challenges/14641531">here</a>.</p>
<p>For sake of sharing good books to check out, here are my top books from 52 or so I read:</p>
<ul>
<li><strong>23 Things They Don’t Tell You about Capitalism</strong> by Ha-Joon Chang - This book offers a different take on economics and economic development. If you are looking to shake assumptions and have good conversations on economic topics, I highly recommend this book no matter what side of the political fence you sit.</li>
<li><strong>Alexander Hamilton</strong> by Ron Chernow - There were multiple moments in reading this book where I just smiled at how interesting of a character Hamilton was. Brilliant, hardworking, luck and flawed, all in one. I gleaned a new perspective on various moments in the early history of the United States.</li>
<li><strong>Stealing Fire: How Silicon Valley, the Navy SEALs, and Maverick Scientists Are Revolutionizing the Way We Live and Work</strong> by Steven Kotler and Jamie Wheal - This book is all about various drugs and technologies being used in pursuit of human augmentation and mental expansion.</li>
<li><strong>The Rise of Superman: Decoding the Science of Ultimate Human Performance</strong> by Steven Kotler - This book is all about flow and how various extreme sports athletes cultivate in pursuit of excellence and high performance. The writing is great throughout but the examples get a bit one-dimensional since everyone is essentially doing high fatality sports. Would benefit from more realistic examples that don’t involve such a high probabily of death.</li>
<li><strong>The Meaning of Life</strong> by Terry Eagleton - This short book starts off quite philosophical around questions of the legimacy of the question itself. It then turns more open ended and offers an all-around good read for anyone wanting well-writing and well-thoughtout philosophical enquiry for the common man philosopher.</li>
<li><strong>Flourish: A Visionary New Understanding of Happiness and Well-Being</strong> by Martin E.P. Seligman - This is a classic and approachable read on the positive psychology. Using validated research, Seligman offers various ways to not just avoid depression but to cultivate a happy and meaningful life of human flourishing. Writing and style is a bit clumsy at times but easy to skim over sections that don’t resonate and still come away with practical takeaways.</li>
</ul>
<h3 id="a-selection-of-my-favorite-articles-from-2019">A Selection of my favorite articles from 2019</h3>
<p>I started a very occassional newsletter to my friends and blog readers in 2019. So if you want to get an informative email every 2-4 months on self-tracking tech and my favorite articles and books, <a href="TODO">subscribe here</a>.</p>
<p>Here are a few of the most memorable articles I read in 2019 and merit a read if you missed them:</p>
<ul>
<li><a href="https://www.technologyreview.com/s/614666/ai-machine-learning-music-feel-good/">AI could help us unpick why some songs just make us feel so good</a></li>
<li><a href="https://getpocket.com/explore/item/america-s-invisible-pot-addicts">America’s Invisible Pot Addicts - The Atlantic</a></li>
<li><a href="http://www.bbc.com/future/story/20190326-are-we-close-to-solving-the-puzzle-of-consciousness">Are we close to solving the puzzle of consciousness?</a></li>
<li><a href="https://psmag.com/news/authoritarianism-the-terrifying-trait-that-trump-triggers">Authoritarianism: The Terrifying Trait That Trump Triggers</a></li>
<li><a href="https://www.goinvo.com/features/from-bathroom-to-healthroom/">Design Thinking: From Bathroom to Healthroom</a></li>
<li>
<table>
<tbody>
<tr>
<td>[Mindfulness is loaded with (troubling) metaphysical assumptions – Sahanika Ratnayake</td>
<td>Aeon Essays](https://aeon.co/essays/mindfulness-is-loaded-with-troubling-metaphysical-assumptions)</td>
</tr>
</tbody>
</table>
</li>
<li><a href="https://www.ncbi.nlm.nih.gov/books/NBK38353/">Meta-Review: Meditation Practices for Health - NCBI Bookshelf</a></li>
<li><a href="https://www.newyorker.com/magazine/2019/04/08/the-day-the-dinosaurs-died">The Day the Dinosaurs Died</a></li>
<li><a href="http://www.grubstreet.com/2018/03/ultimate-conversation-on-healthy-eating-and-nutrition.html">The Last Conversation You’ll Ever Need to Have About Eating Right</a></li>
<li><a href="https://politi.co/2I7zNfE">The Secret History of Fort Detrick, the CIA’s Base for Mind Control Experiments</a></li>
<li><a href="https://www.theatlantic.com/magazine/archive/2019/05/the-trouble-with-dentistry/586039/">The Trouble With Dentistry</a></li>
<li><a href="https://www.vanityfair.com/style/2014/05/super-rich-perfection-anxiety">What Is It Like to Have Too Much Money?</a></li>
<li><a href="http://www.atlasobscura.com/articles/pet-squirrel-craze">When Squirrels Were One of America’s Most Popular Pets</a></li>
<li><a href="https://www.theatlantic.com/magazine/archive/2018/12/the-sex-recession/573949/">Why Are Young People Having So Little Sex?</a></li>
<li><a href="http://getpocket.com/explore/item/why-is-the-human-brain-so-efficient">Why Is the Human Brain so Efficient? - Nautilus</a></li>
<li><a href="https://www.bbc.com/worklife/article/20191118-what-shapes-a-polymath---and-do-we-need-them-more-than-ever">Why some people are impossibly talented</a></li>
</ul>
<h5 id="looking-ahead">Looking Ahead:</h5>
<p>I again set a goal of reading 52 books in 2020. This should keep me on pace for my <a href="http://www.markwk.com/2016/06/2000-book-reading-challenge.html">2000 book reading challenge</a>. I have several areas I want to learn and deepen in year ahead, so I plan look more into fields like neuroscience, react native and machine learning.</p>
<p>I still find information integration and learning synthesis to be an area I can improve in year ahead. I am exposed to a lot of great ideas and concepts, and, while I do a good job taking notes (more on this below), this would be an area I can expand upon in pursuit of <a href="http://www.markwk.com/science-of-learning.html">learning how to learn</a>. The fact is that if an idea is important <em>to me</em>, I need to take notes and bring these notes in my <a href="http://www.markwk.com/plain-text-life.html">smart notes system</a>.</p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}This data was logged with Goodreads, Kindle and Instapaper. Data collection and visualization powered by QS Ledger. This post is part of my 2019 Year in Data project.A Year in Health and Fitness: 20192020-02-02T00:00:00-08:002020-02-02T00:00:00-08:00http://www.markwk.com/year-in-health-fitness<h5 id="how-i-tracked-it">How I Tracked It:</h5>
<p><a href="http://www.markwk.com/tracking-health-or-habits.html">Are you tracking a health indictor or healthy habit?</a> As I recount in <a href="http://www.markwk.com/health-trackers.html">Data-Driven Health Trackers: An Actionable List</a>, there are a number of ways to quantify our health and bodies.</p>
<p>Over the last year and for this year’s fitness report, I logged and collected my data through multiple tools:</p>
<ul>
<li><strong>Apple Watch: Steps and Sleep</strong>: I use <a href="http://www.markwk.com/apple-watch-for-self-trackers.html">Apple Watch</a> to track my steps as well as wear my watch to bed to <a href="http://www.markwk.com/sleep-tracking-tools.html">passively track my sleep</a> using AutoSleep app. It may not be the most advanced wearable or biohacker gadget like an Oura Ring, but the ubiquity of Apple products and the realiable of the Apple Watch in particular have made it one of my favorite tracking tools.</li>
<li><strong>Strava: Running and Cycling</strong>: I also use my watch with GPS to record my <a href="http://www.markwk.com/category/running/">running and cycling</a> with Strava.</li>
<li><strong>Mobility and Stretching with Apple Health Kit Shortcut</strong>: I <a href="http://www.markwk.com/mobility-training-stretching-tracking.html">record my stretching and mobility</a> with a shortcut and Apple Health</li>
<li><strong>Strength Traing with Fitbod</strong>: I manage my strength training with the app, Fitbod, that helps me figure out which exercises to target which muscles.</li>
<li><strong>Heart Variability / Chronic Stress</strong>: When it comes to <a href="http://www.markwk.com/hrv-for-beginners.html">Heart Rate Variability</a>, one of my favorite biomakers and a measure of stress, I use HRV4Training, which not only tells me my morning readiness but also gives me insights into things that negatively affect my HRV like lack of sleep, travel and drinking. For all of 2019 I logged my HRV with the camera app on the phone but for 2020 I recently switched to collecting this on my wrist using Apple Watch’s Breathe app.</li>
</ul>
<p>Data collection and visualization powered by <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>. This post is part of my <a href="http://www.markwk.com/category/year-in-data/"><em>2019 Year in Data</em> project</a>.</p>
<h5 id="highlights">Highlights:</h5>
<p>I’m staying active!</p>
<p>After a decade of physical inactivity (and the weight to show for it), a few years ago I’ve managed to kickstart a couple of positive health and fitness habits. Personally I relied on self-tracking and data visualization along with goal setting and goal check-in’s to ensure I stuck to my commitments and gauged my progress. Whether or not wearables and tracking work for everyone (or at all as some researchers have claimed) as some have claimed, health data provides a useful feedback loop towards positive lifestyle changes.</p>
<p>I currently stick to a number of simple healthy fitness habits. I set time in my schedule for running and strength training, and most of the time that means I’ve protected some time to run a few times a week and even go to the gym for strength training.</p>
<p>In 2019, I completed a couple half-marathons but no full marathons like I did in 2017 and 2018. One biggest running highlights of the year was finishing my first trail race. In November I completed the Griffith Park Half Marathon Trail Race in Los Angeles. Since I was actually training for a road race, I didn’t get much hill training beforehand, but I finished the race in the top 25% and enjoyed amazing views, nutritious snacks and good community throughout.</p>
<h5 id="what-i-learned">What I Learned:</h5>
<ul>
<li><strong>Sleep</strong>: Without the data I would have thought I had slept roughly the same in 2018 and 2019 but the logs beg to differ. In fact, I slept a good deal less in 2019 than I did the year before. <strong>Roughly equivalent to 10 minutes less per night or roughly 60 hours less in the year!</strong> According sleep researcher Matthew Walker in “Why We Sleep,” chronic sleep loss can be just as bad cognitively and for our health as a single night or two of sleep deprivation. Lack of sleep has an health impact too, leading to a higher probability of infection and sick days. But for someone like me who depends on creativity and learning, lack of sleep leads to worsening cognitive performance.</li>
<li><strong>Running</strong>: I did less running compared to 2018 (918km vs. 1559 kms). Sickness and slight injuries made the first half of 2019 a hard year for my running and races. I wonder if lack of sleep, less mobility work, and increased travel and life stress contributed to this? The data seems to indicate that. It’s well-known that less training leads to a lower <a href="http://www.markwk.com/2017/07/finding-my-vo2-max.html">VO2 Max</a> and worsening performance.</li>
<li><strong>Strength Training</strong>: One big change was adding strength training into my weekly health and fitness routines. I got a gym membership (in China, Taiwan and Los Angeles) and went to the gym and lifted consistently for several months last year. I felt better, stronger and my running was less pained. Some research indicates that strength training makes runners less injury-prone too. Heavy periods of travel, especially in Q4, made it hard to lift.</li>
</ul>
<h5 id="looking-ahead">Looking Ahead:</h5>
<p>I’ll admit that I used fitness goals and hobbies in the past as a way to replace or deal with lack of progress in other parts of my life. It’s nice that to have a sense of control. It feels good to see progress in your fitness even if your startup project, relationship or something eles is floundering. But focusing on fitness goals can also be a distraction from your true passion and purpose.</p>
<p>As I wrote in my own marathon story, <a href="http://www.markwk.com/first-marathon.html">Don’t Turn Left at 24km: My Story Training and Running the Chengdu Panda Marathon</a>, I highly recommend training and running a marathon, since it teaches displine and is a powerful way to show just how adaptable we are. This is particularly important for entrepreneurs and creatives who struggle with self-doubt, imposter sydrome and failure in the face of so many uncertainities. Marathon running and other epic physical challenges teach you a lot of transferrable life lessons.</p>
<p>That said, personally and at least for the year ahead, I no longer place heath and fitness goals at the top of my priorities. They remain important but just not the cornerstone of my life goals. Instead, <em>health and fitness are just what we do so we can pursue excellence in other areas</em>. In my case that means creating, writing and learning. Good health is essential for peak performance.</p>
<p>So looking ahead, my broad and quantifable goal is to stick to getting about <strong>3-5 hours of formal exercise per week</strong>. This exercise number corresponds to what most of the reseachers consider the most beneficial for good health and about longevity. Breaking this down, I’m aiming at 2-3h or 20-30km a week of running and ideally 1 to 2.5h / week of strength training. As I wrote about in <a href="http://www.markwk.com/data-driven-run-training.html">Data-Driven Run Training</a>, I use TrainAsOne as my AI training plan coach, since it does a great job of balancing volume and run types towards my training and race goals and avoiding overtraining. When it comes to lifting (a topic I haven’t written much on yet!), I do core lifts (deadlift, squat and bench press) and have used Fitbod app to track my workouts and keep my workout routine fresh.</p>
<p>I still like running races, especially as a way to discipline my training and give me a few peak events per year. They are a lot of fun too! I’ve signed up for the LA Marathon in early March and currently in training mode.</p>
<p>Since I’m now living in Venice, California, cycling and walking are now primary means of transportation. This is a nice way to automatically stay active. Now that I’m settled into a few routines and weekly meetings, it’s cool to see an additional 2-4 hours of week cycling just as I go about my day!</p>
<p>While I eat pretty well and get “adequant” amount of sleep, I still think there is room for improvement on both. Unfortunately poor sleep and travel likely led to a long sickness I had at the start of 2019 as well (and also why this year in data post is so delayed!). Personally, I’d like to get my daily sleep average closer to 8 hours per night and decrease my drinking to just once or twice a week.</p>
<p>I eat a primarily plant-based diet and take a couple supplements, like Creatine and Krill Oil. I fast quarterly. It might be time to research and “biohack” my fasting and supplementation in the months ahead with a eye towards <a href="http://www.markwk.com/blood-test-faq.html">blood biomarker tracking</a> and longevity.</p>
<p>Ethically and ecologically, I’d like to cut my meat consumption down to just once or twice a week and instead eat more beans and legumes for protein. Food tracking is hard but it might be interest to simply track how I eat meat, fish and dairy.</p>
<p>I’m currently doing a major food tracking and glucose study with a continuous glucose monitor (CGM), so I suspect I’ll have more nutrition-related insights to incorporate and write about soon. Even without a full data analysis I can tell you that IHOP Strawberry Pancakes led to one of my highest glucose spikes!</p>
<p>Compared to my other tracking areas like <a href="http://www.markwk.com/2020-year-in-time.html">time</a>, <a href="http://www.markwk.com/2020-year-in-tasks-goals.html">tasks</a> and writing, health and fitness tracking have yet to get as robust of a personal data analysis. This is partially due to how complicated to do data analysis on our health. Human physiology and biology are complex system. It is difficult track how food, sleep and exercise affect us day to day, let alone to understand these effects over longer expanses of time. I definitely hope to work more on my own <a href="http://www.markwk.com/category/data-driven-health/">data-driven health</a> and see what my health and fitness data says in the year ahead.</p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}How I Tracked It:A Year in Tasks, Projects and Goals: 20192020-01-31T00:00:00-08:002020-01-31T00:00:00-08:00http://www.markwk.com/year-in-tasks<p>This data was logged with <a href="http://www.markwk.com/task-tracking-with-todoist.html">Todoist</a> and <a href="http://www.markwk.com/goal-scoreing.html">my simple goal scoring system</a>. Data collection and visualization powered by <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>. This post is part of my <a href="http://www.markwk.com/category/year-in-data/"><em>2019 Year in Data</em> project</a>.</p>
<h5 id="how-i-tracked-it">How I Tracked It:</h5>
<ul>
<li><strong>Task Tracking</strong>: As I wrote about in <a href="http://www.markwk.com/task-tracking-with-todoist.html">Tracking Your Tasks with Todoist</a>, I use Todoist as my task management tool. I’ve used various apps, including <a href="http://www.markwk.com/gtd-with-evernote.html">Evernote</a> for a few years until I <a href="http://www.markwk.com/migrate-evernote-plaintext.html">migrated off of Evernote</a> a year ago. Todoist helps me to manage my upcoming actionable items for work and personal projects. Todoist is a well-designed product that is built in the spirit of <a href="http://www.markwk.com/gtd-with-todoist.html">Getting Things Done</a>. I’m a fan of how I can assign Todo items in my calendar from Todoist. When it comes to data collection, their API is great but you can also use IFTTT and create a recipe so all of your completed tasks are saved into a Google Spreadsheet.</li>
<li><strong>Goal Scoring</strong>: One new addition this year was Goal Scores, which I wrote about in <a href="http://www.markwk.com/goal-scoreing.html">Scoring Your Weekly Goals</a>. The basic idea is each week I set one or two major objectives and a few minor ones. For me, I score major goals as worth 1.5 each, and minor objectives are scored by percentage completed multipled by 3. So the max score would be 6. I find this to be a solid way to make weekly progress and track that progress in a relatively objective way. My goal scores showed a significant drop in May when I was on a work holiday in the US.</li>
</ul>
<h5 id="highlights">Highlights:</h5>
<p>I got a lot accomplished in 2019 in terms of tasks and personal goals. Unfortunately, a couple months of travel and my move to Los Angeles both contributed to a less stable work and life situation. The uncertainty and stress of moving meant I got less tasks done at several stages of the year. Setting up my personal and professional life in a new place also meant I spent a lot more time on networking and meetings, which the data clearly shows. I had a high concentration on networking from Sept to mid-November. This came through a combination of a job hunt and looking for new clients. Like my <a href="http://www.markwk.com/2020-year-in-time.html">time logs</a>, there are several noteable patterns, namely we see consistent organizational work on each and nearly every Sunday or Monday when I do my <a href="http://www.markwk.com/data-driven-weekly-review.html">weekly review</a>.</p>
<h5 id="what-i-learned">What I Learned:</h5>
<p>Before doing this year in data, I largely would have thought I spent roughly the same amount of time and completed roughly the same number of tasks in 2019 as 2018. Our retrospective memory struggles to estimate such large expanses of time. It’s possible that the productive end of the year biased me to think I was more productive across the year. My actual data showed this wasn’t the case. My completed tasks was less than last two years. I still largely completed all of my major yearly goals, but I didn’t check off as many things from my todo lists over time.</p>
<h5 id="looking-ahead">Looking Ahead:</h5>
<p>Looking at my task data makes me think that 2019 was a somewhat inconsistent year in terms of my productivity. In view of the fact that I traveled more and moved countries, this shouldn’t have been such a surprise. I was still getting a minimum amount done, but it was less than in previous years.</p>
<p>So my main takeaway looking forward is to ensure greater consistency in my working pursuits. I need to strive at sticking to a schedule where I get both minor and major tasks completed each and every day. Personally I’ve found <a href="http://www.markwk.com/goal-scoreing.html">Weekly Goals and Goal Scoring</a> to be a simple and powerful way to get key things done (and also track it). I find that I can use my yearly and monthly goals to align weekly objectives so I do more work that aligns with my passions and long-term objectives. In the year ahead, I’d like to avoid low productivity days.</p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}This data was logged with Todoist and my simple goal scoring system. Data collection and visualization powered by QS Ledger. This post is part of my 2019 Year in Data project.A Year in Time: 20192020-01-30T00:00:00-08:002020-01-30T00:00:00-08:00http://www.markwk.com/year-in-time<p>This data was logged with RescueTime and Toggl. Data collection and visualization powered by <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>. This post is part of my <a href="http://www.markwk.com/category/year-in-data/"><em>2019 Year in Data</em> project</a>.</p>
<h5 id="highlights">Highlights:</h5>
<p>While I spent less time on my computer in 2019, I still spend a lot of time computing with several days where I spent 10+ hours. Overall, I still have gained a lot more balance in how much time I spend on the computer compared to previous years. I’m happy that I don’t spend the majority of my computer time on email and instead most of my time is on writing, note-taking and software development. It’s also great that I maintain a consistent ratio of computer time to project time, which basically means that when I’m on the computer, I’m working on a project or goal. Sadly, I still spend a lot of time on non-computer devices.</p>
<h5 id="how-i-tracked-it">How I Tracked It:</h5>
<p>I continue to <a href="http://www.markwk.com/time-tracking-tools.html">track my time</a> primarily using two tools, RescueTime and Toggl. Toggl lets me record my project time and it’s a <a href="http://www.markwk.com/2015/09/in-my-time.html">conscious way to notice what I’m working on</a>. RescueTime is an entirely passive way to track your computer usage. Interestingly, I’ve used both of these tools since 2013 and this was the source of one of my early year in data visualizations, <a href="http://www.markwk.com/2016/01/a-year-of-time-tracking-2015.html">How I Tracked a Year in Time and What It Meant</a>. For device and screentime tracking, I use Apple’s Screentime and simply log this weekly into a spreadsheet. This provides a pretty easy way to see my evolving device usage over time.</p>
<h5 id="what-i-learned">What I Learned:</h5>
<p>Besides increasing the number of “computer free” days, the two biggest lessons from looking at my time tracking from this last year were:</p>
<ul>
<li><strong>Mobile Phone and Tablet “Addiction”</strong>: Like most everyone I struggle with my screetime, meaning my phone and tablet usage. Digital wellness is something we all need to grapple with. While improvements have been made, I still feel like now and this past year I spend far too much time on my phone. This is partially because I’m a <a href="https://int3c.com/">mobile app developer</a>, but it’s also because I still have plenty of time sucks on my phone. Even though I don’t use Facebook, Twitter or Instagram, I have things that pull me into a lot of screen time. For me the two biggest time sucks on Netflix and YouTube, both of which I use on my phone or tablet. This year I did a <a href="http://www.markwk.com/youtube-30-day-challenge.html">30-Day “No YouTube” Challenge</a> that helped me gain perspective and decrease my usage (at least for awhile). One idea would be to get more aggressive with using Airplane mode or Disabling Notifications during certain hours.</li>
<li><strong>My Evolving Obsessions</strong>: As a <a href="http://int3c.com">product manager and entrepreneur</a> for the last decade, I’ve tended to work in cycles across the year. One or two projects will pull at my attention and I’ll put in considerable time and tasks on that for awhile until once completed or as new opportunities shift my focus, I’ve vaguely known this for year, but my time logs and visualizations have made these patterns more clear. It’s interesting to see in the heatmaps how visiable certain evolving obcessions become over days and weeks. For example, we see client dev work clustering in the beginning and end of the year, while my studies and writing had a strong push over the summer. This pattern is also repeated in my task logs (see below).</li>
</ul>
<h5 id="looking-ahead">Looking Ahead:</h5>
<p>While I wish I followed a single schedule year round, I doubt this will ever happen. This is just the nature of my work and how much of the consulting and startup space operates. I need to continue to embrace a certain core of routines and habits, namely around time management and morning routine. I need to ensure I read, study and write consistently (over less productive phone and tablet use). Having blocks in my schedule for writing or code studies makes it clear to me what matters.</p>
<p>Overall, I will continue to work in batches with targetted outcomes on client projects, personal goals and new products. Basically it’s about developing organizational processes and a schedule so I do the fundamentals (writings, running, etc.) and finding daily blocks to pursue deep work and cultivate flow.</p>
<p><br /></p>
<p><a href="http://www.markwk.com/category/year-in-data/">Check out other posts, data visualizations and infographics from my year in data!</a></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}This data was logged with RescueTime and Toggl. Data collection and visualization powered by QS Ledger. This post is part of my 2019 Year in Data project.Paper to Paperless: A Guide to Digitalizing Your Journals with a Scanner App2019-10-09T00:00:00-07:002019-10-09T00:00:00-07:00http://www.markwk.com/digitalizing-paper-journals<p>One of the most effective and powerful tools for taking notes, capturing memories and creativity in general is using a physical notebook or journal. Research has shown that writing by hand improves our retention and learning and even has benefits for our imagination too. While typing and digital photos might be the main mode most of us work in, I am increasingly convinced that in my own life about utility of a hand-written journal. Furthermore, it’s a key creative habit that all creatives from writers and artists to programmers and data scientists should consider using.</p>
<p>But what’s to be done with the notebook afterwards? Once we are done with it, should we throw it away, stow it in a box or on a shelf, or might we do something else with it? Perhaps we can preserve it in another way? Perhaps we can take take our paper journals and digitalize them?</p>
<p>While you may not want to go to the lengths of Microsoft engineer Gordon Bell, who <a href="http://www.markwk.com/most-digitalized-life-ever.html">scanned and archived almost every single aspect of life</a>, I’d argue that once we can and should take our paper notebooks and make them digital. Unlike paper, digital files won’t mold and degrade. They can be easily stored in the cloud or to a digital backup so they can last forever. With digital files in the cloud, you can access them anywhere with an internet connection and share them too. Digital files are easy to reorganize so you can mix-and-match them too. Additionally by adding some textual annotations or a table of contents, it’s fast to find your pages from a certain date or on a particular topic.</p>
<p>Ultimately the principal benefits of a paper journal are in the moment. You capture things when and where you are. For example, you <a href="http://www.markwk.com/smart-notes.html">read with a pen</a>, which allows you to <a href="http://www.markwk.com/smart-notes.html">take better or smarter notes</a>. You might also make a sketch, note keywords, write-down quotes, create lists, or make outlines. These are all powerful tools for learning and creating with a physical notebook, but there is benefit in digital files too. In most cases your final production or output will be digital. So it’s great to have your paper notebooks incorporated into your digital archive.</p>
<p>Paper and pen are powerful tools, but eventually you want you want to go digital and live paperless.</p>
<p>Fortunately, by using either an actual flat-bed scanner or, as I’ll show here, a scanner app on your phone, it’s easy to scan and digitalize your paper journals as well as photos, letters and anything less on paper. For example, you can digitalize an entire paper journal of 100+ pages in 20 or 30 minutes. Additionally, with a couple extra steps you can add annotations or append a table of contents to help you keep things better organized and link paper pages to your digital doings. .</p>
<p>In this post, I want to share a small slice of how to digitalize a life. Unlike <a href="http://www.markwk.com/most-digitalized-life-ever.html">Gordon Bell and his epic quest to digitalize everything</a>, our target is a bit more modest: scanning and digitalizing our journals and notebooks. In the first paper, I provide an overview of the tools and techniques for scanning with either a flat-bed scanner or an app as well as how to add a table of contents. After that, in the the second part, I’ll share a quick example of scanning one of my own notebooks and tell you just how long it took to do. Finally, I’ll conclude with a few general thoughts on the benefits of digitalizing your notebooks, lifelogging and incorporating them in your digital archive.</p>
<p><br /></p>
<p><em>NOTE: This post covers the practical aspects to digitalizing a paper notebook or journal. For a bit of history and background, see my previous post, <a href="http://www.markwk.com/most-digitalized-life-ever.html">Gordon Bell and The Epic Quest to Digitalize Everything</a>.</em></p>
<!--more-->
<h2 id="a-modest-how-to-guide-for-going-paperless">A Modest How-To Guide for Going Paperless</h2>
<p>The end goal of digitalizing a journal is to have a single file showing an image of each physical page and a table of contents that lets you look up past entries quickly.</p>
<p>There are a few tools you’ll need for digitalizing a life in paper:</p>
<ul>
<li><strong>Scanning</strong>: Flat-bed Scanner or Scanner App</li>
<li><strong>TOC / Annotations</strong>: PDF Editing Tool like Acrobat or Preview and/or some basic coding skills.</li>
</ul>
<p><br /></p>
<h3 id="physical-scanners-highest-quality-way-to-go-digital">Physical Scanners: Highest Quality Way to Go Digital</h3>
<p>A flat-bed scanner is arguably the best way to turn physical journals (or photos and letters for that matter) into digital documents. This was the method used by Gordon Bell in his own digitalization project.</p>
<p>Today there are even some scanners that can be loaded with multiple photos, papers or a book and automatically scan multiple pages at once. This was how Google managed to digitalize many public domain books too.</p>
<p>Ultimately, an actual scanner will provide the best fidelity in capturing physical objects digitally. For example, my father has been using a scanner in his own archive project digitalizing photos and docs from his (including his <a href="https://www.flickr.com/photos/larrywkoester/45198267155/in/album-72157704099291084/">first grade report card</a>!).</p>
<p>Though there is a wide range of prices and features, you can generally buy a good photo and document scanner for around $100 usd. Check out <a href="https://www.pcmag.com/roundup/250166/the-best-photo-scanners">PCMag</a> or <a href="https://bestreviews.com/best-scanners">Best Reviews</a> for some reviews of the most popular models.</p>
<p>Unfortunately, the process of scanning and digitalizing with a physical scanner can be rather tedious and time consuming. You must load the document, paper or photo into the scanner and do the scan which can take a few minutes. Then on the computer, you will likely need to use some software to correct the final images too. Auto-corrections for contrast and straightening works great in most cases, but it does take time.</p>
<p>One thing to also note about physical scanners is that you need to check and clean them periodically. Smudges and dust can built up and degrade your scan quality.</p>
<p>If your objective is to get a digital version of your notebooks that can be printed at high quality, then a physical scanner is the way to go. But if you are merely looking for readability and general accuracy for personal usage in your scans, then an app might be a faster method to try.</p>
<h3 id="scanner-apps-the-quick-and-efficient-way-to-go-paperless">Scanner Apps: The Quick and Efficient Way to Go Paperless</h3>
<p>If you are looking to quickly and with less effort digitalize your notebooks, arguably your best bet is a scanner app. A scanner app uses software to optimize either digital images you’ve already taken or uses your mobile phone’s camera to act like a scanner. Basically it improves the photos you take so they are sharper and flatter and can be converted into documents.</p>
<p>Like photography and camera apps in general, scanner apps are one of the more popular categories in iOS and Android app stores. There are dozens, perhaps hundreds of apps to choose from. Some of the top options that come up are Scanbot, Adobe Scan, Microsoft’s Office Lens, CamScanner, Tiny Scanner, and Google’s PhotoScan. All of which I have tried and worked fine.</p>
<p>Most work great for scanning needs, come with various features and sharing options. Unfortunately Scanbot Pro and Adobe Scan both require a subscription now, so likely they aren’t a good choice for casual or budget-conscious users.</p>
<h5 id="genius-scan-my-preferred-document-scanner-app">Genius Scan: My Preferred Document Scanner App</h5>
<p>Based on my own testing and extensive usage on iOS, my personal pick of Scanner Apps is <strong><a href="https://www.thegrizzlylabs.com/genius-scan">Genius Scan</a></strong>, which is is also available for Android too.</p>
<p><img src="https://static1.squarespace.com/static/578fc05359cc681e6e24dc11/t/5b8797474ae23721f5d3e938/1535612814018/Scan+documents" alt="Instantly scan documents - Genius Scan smart algorithms automatically detect your document, apply perspective correction and enhance the colors." /></p>
<p><small>Source: https://www.thegrizzlylabs.com/genius-scan</small></p>
<p>Genius Scan does a great job of automatically detecting document edges while you take photos using the camera or while directly editing static images. The app then resizes and positions the images accordingly to create a flat, document view. You can apply appropriate filters so docs are high contrast and black and white (or alternatively leave them as colored photos). You can also manually edit scans to correct misapplied edge detection, use an alternative color filter, or make other corrections.</p>
<p><img src="/generated/images/2019-resources/scanned-journal-cover-500-0c5564ced.jpg" srcset="/generated/images/2019-resources/scanned-journal-cover-400-0c5564ced.jpg 400w, /generated/images/2019-resources/scanned-journal-cover-500-0c5564ced.jpg 500w" /></p>
<p>You can take single pics or use multiple pictures to create a single scanned document. This feature works really well and makes it time-efficient if you want to scan a book, a series of individual pages, or even multiple receipts from a trip in a single session.</p>
<p>Once you’ve scanned and editing the documents to your liking, Genius Scan lets you export the entire document as a PDF and put it where you want it. For example, you can share it by email, save it to a cloud storage like Dropbox or Google Drive, or store it locally on the device. You can also select individual scans and export as a JPEG.</p>
<p>For general usage, open up the app and tap the PLUS button and start scanning pages using the camera. It should detect edges and start snapping pics for each page. Once complete, export it to a PDF.</p>
<p>The end result: You’ve transformed physical paper into digital!</p>
<p><small>NOTE about Example Scan: This is one of my earliest notebooks from 2004-08-to-2006-03 following gradation in Chicago and moving to Europe.</small></p>
<h3 id="adding-a-table-of-contents-toc-to-a-pdf">Adding a Table of Contents (TOC) to a PDF</h3>
<p>While scanning and digitalizing your journal is the most important part of this process, I find that adding annotations and a table of contents (TOC) makes your scanned journals even more usable, better organized and powerful. Without a table of contents, you are forced to scan page by page to find what you are looking for. But by appending a digital table of contents, you can quickly look up any of the main sections with just a single click.</p>
<p>There are a few different options for creating a table of contents on a PDF document. The most common way we normally create a TOC is by first putting links into the Word document we are creating and then exporting it all to a PDF. In this way the TOC should automatically be included. Unfortunately, for our needs, we already have a PDF so we need to add a table of contents to an existing document.</p>
<h5 id="toc-editing-software">TOC Editing Software</h5>
<p>Since we already have a PDF and need to create or edit the TOC, our options are a bit more limited. One option is to use special PDF editing software to do this. Three options include:</p>
<ul>
<li>Adobe Acrobat Pro</li>
<li><a href="https://www.onekerato.com/pdfoutliner.html">PDFOutliner</a></li>
<li>Foxit Reader</li>
</ul>
<p>Each of these provides a way to add or edit existing TOC in your PDFs. Unfortunately at the time writing, none of these are free.</p>
<h5 id="appending-a-toc-using-a-text-file-and-a-python-script">Appending a TOC using a Text File and a Python Script</h5>
<p>One alternative to add a table of contents to your scanned journal and PDF document is to use a python script I’ve created and shared <a href="https://gist.github.com/markwk/c0157ffce0ed0308c7647ab387986908">here</a>. It requires a bit of coding and is slightly more technical way but it is also free and customizable.</p>
<p>Here’s how to do it.</p>
<p>On Mac, you should download or copy you script from <a href="https://gist.github.com/markwk/c0157ffce0ed0308c7647ab387986908">Github Gist</a> and paste the code into a file called something like <code class="language-plaintext highlighter-rouge">pdf_toc_processor.py</code>.</p>
<p>After that, you need to create a text file of your bookmarks or table of contents like the following example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1, Cover
2, August 20 2018 @ Singapore
4, August 22 @ Los Angeles
</code></pre></div></div>
<p>The number is the page you will link to do and the text is the title of the bookmark itself.</p>
<p>After creating your PDF file and preparing your bookmarks / TOC, you’ll need to run the main command:</p>
<p><code class="language-plaintext highlighter-rouge">python pdf_toc_processor.py -i <path-to-target.pdf> -b <path-to-bookmarks-file.txt> -o <path-to-output.pdf></code></p>
<p>This command requires three input paths: initial PDF, bookmarks file, and output PDF.</p>
<h2 id="complete-step-by-step-guide-to-digitalizing-a-physical-journal-with-a-scanner-app">Complete Step-By-Step Guide to Digitalizing a Physical Journal with a Scanner App</h2>
<h5 id="scanned-notebook-to-pdf-with-digital-toc">Scanned Notebook to PDF with Digital TOC</h5>
<p><img src="/generated/images/2019-resources/pdf-append-table-of-contents-example-800-1e3692b3e.jpg" srcset="/generated/images/2019-resources/pdf-append-table-of-contents-example-400-1e3692b3e.jpg 400w, /generated/images/2019-resources/pdf-append-table-of-contents-example-600-1e3692b3e.jpg 600w, /generated/images/2019-resources/pdf-append-table-of-contents-example-800-1e3692b3e.jpg 800w, /generated/images/2019-resources/pdf-append-table-of-contents-example-1000-1e3692b3e.jpg 1000w" /></p>
<p><small>Example of a recent journal I scanned with the bookmarks file and appended TOC in the PDF.</small></p>
<p>For me there are two primary usages for scanner apps.</p>
<p>The first and most common use case for me is scanning receipts and various paper items I collect on trips. While it’s possible to just use your camera to do this, a dedicated app makes it easier to key everything in one place and generate a final PDF export you can share with your company or keep for accounting purposes.</p>
<p>The second reason I have for scanning was in the spirit of Bell’s MyLifeBits project: I wanted to digitalize and create an archive of my assorted journals and notebooks.</p>
<p>I’ve been writing regularly in some form since I was in high school. Though it has ebbed and flowed, I’ve basically always had some sort of notebook for jotting ideas, sketching and writing. Over the years it’s grown to quite a pile.</p>
<p>TODO: PHOTO HERE OF JOURNALS</p>
<p><small>Some 20 or so odd physical journals I’ve created over the years. Is it time to digitalize?</small></p>
<p>As a minimalist and digital nomad, it doesn’t make sense to lug around an extra bag just for my old paper-bound scribblings that I rarely need anyways. In fact, I mostly use my notebooks as an initial step towards digital <a href="http://www.markwk.com/smart-notes.html">smart notes</a> as part of my <a href="http://www.markwk.com/plain-text-life.html">plain-text file archive</a> anyways. Instead I wanted to create a digital copy for my archive and and ideally make it annotated too.</p>
<h5 id="step-1-setup-for-scanning">Step 1: Setup for Scanning</h5>
<p>Here are two considerations for I follow in digitalizing my journals:</p>
<ol>
<li><strong>Find a Good Scanning Surface</strong>: Ideally you want it to be non-white with a high contrast compared to the material you are scanning. While a bed might work, I got the best results with a desktop or table. The aim should be a none-white surface so the app can detect edges better.</li>
<li><strong>Check and Adjust the Lighting</strong>: The biggest issue you will likely confront in getting a good scan using your phone are shadows. The lighting in the room can cause your phone or arm to put a darker shadow on the page. Ideally find a desktop lamp or setup during the day where you minimize or eliminate unwanted shadows. Most apps can deal with it but you’ll see better results without shadows.</li>
</ol>
<h5 id="step-2-test-scans">Step 2: Test Scans:</h5>
<p>Once the space and lighting are ready, fire up your scanning app and do a few test scans. Before you scan an entire notebook, it’s best to do a “reality check” on a few pages to confirm the results are good enough for you needs.</p>
<h5 id="step-3-scan-an-entire-document">Step 3: Scan an Entire Document</h5>
<p>It’s likely you have multiple books you want to scan, and it might take multiple sessions. That said, I recommend starting with scanning one entire journal first and do all of the pages in it. Avoid scanning the same document over multiple sessions since lighting might can leading to an inconsistent final product.</p>
<p>Once everything looks good in testing, then just start scanning.</p>
<p>Depending on the app, it might attempt to detect edges while you take photos. If it doesn’t detect it right the first time, I recommend just continuing to take photos of the complete document rather than editing while you shoot. Take advantage of batching effect and focus on capturing all of the pages in one session.</p>
<p>Personally, even before editing and in spite of a few pages that had poor edge detection, I was quite pleased with the results:</p>
<p><img src="/generated/images/2019-resources/scanned-journal-page-example-800-8cee17fae.jpg" srcset="/generated/images/2019-resources/scanned-journal-page-example-400-8cee17fae.jpg 400w, /generated/images/2019-resources/scanned-journal-page-example-600-8cee17fae.jpg 600w, /generated/images/2019-resources/scanned-journal-page-example-800-8cee17fae.jpg 800w, /generated/images/2019-resources/scanned-journal-page-example-1000-8cee17fae.jpg 1000w" /></p>
<h5 id="step-4-check-and-edit-your-scanned-document">Step 4: Check and Edit Your Scanned Document</h5>
<p>Once you’ve captured all of the pages, it’s time to look at it page by page and edit accordingly. Genius Scan did a good job during the editing process of detecting most edges and making the appropriate corrections, but since the journal I used wasn’t completely flat and there were some colored pages, I needed to do some manual editing.</p>
<p>The editing process with Genius Scan and similar apps is quite straightforward. The main task is to help the app with identifying the edges of the page. So your task is to adjust the rectangle to cover the edges accordingly. Once set correctly, Genius Scan can reposition it for ideal viewing and export.</p>
<h5 id="step-5-export-to-pdf">Step 5: Export to PDF</h5>
<p>Now that the journal pages have all been photographed and correct, you are ready to export it. Like most apps, Genius Scan offers a range of options for export. For my purposes, a simple export to my file directory was enough.</p>
<h5 id="step-6-typing-up-the-table-of-contents">Step 6: Typing up the Table of Contents</h5>
<p>Now that everything has been digitalized, it’s up to you if you want to do anything else. For my purposes, I think taking a few extra minutes to add a table of contents makes the document a lot more usable. Using the script and example I provided above, go through the export PDF and create a text file of the page number and the title or topic of the page. This document can be used not only to create the TOC but it can also be a good reference to include in your own <a href="http://www.markwk.com/smart-notes.html">smart notes archive</a> too.</p>
<h5 id="step-7-run-script-to-append-toc">Step 7: Run Script to Append TOC</h5>
<p>Once you’ve got the initial PDF and a bookmarks TOC, it’s time to run our script:</p>
<p><code class="language-plaintext highlighter-rouge">python pdf_toc_processor.py -i my-journal-2018.pdf -b my-journal-2018-toc.txt -o my-journal-2018-with-toc.pdf</code></p>
<h5 id="final-result-scanned-notebook-to-pdf-with-digital-toc">Final Result: Scanned Notebook to PDF with Digital TOC</h5>
<p><img src="/generated/images/2019-resources/scanned-journal-cover-500-0c5564ced.jpg" srcset="/generated/images/2019-resources/scanned-journal-cover-400-0c5564ced.jpg 400w, /generated/images/2019-resources/scanned-journal-cover-500-0c5564ced.jpg 500w" /></p>
<p>Using my smartphone, Genius Scan and a bit of simple scripting, I created a PDF document that was both extremely high quality and detailed. It had a TOC for easy reference, and I now had a digital version of my notebook forever. For this example, the final size clocked in at 110 meg of space too.</p>
<p>So, how long did scanning and digitalizing this particular notebook take?</p>
<ul>
<li>15 to 20 minutes for capturing via the app all 170+ pages in this particular notebook</li>
<li>20-25 minutes more for additional manual cropping, editing and exporting to PDF.</li>
<li>20-25 minutes to type up the TOC and append to the PDF.</li>
</ul>
<p>Even though a physical scanner might be higher quality, there is no doubt the using a phone and scanner are a faster approach. All told, the scanning and editing work for a single notebook took me about an hour over 3 different sessions. I believe as I do this more, I can slightly decrease the time it takes. That said, considering that I still have about 25 more notebooks to scan, I estimate this will take 20+ hours to complete. Now I just need the actual time to do it!</p>
<h3 id="conclusion-paper-made-digital-from-lifelogging-to-meaningful-digital-archiving">Conclusion: Paper Made Digital: From Lifelogging to Meaningful Digital Archiving</h3>
<p>I’ll admit that I am a strong adherent of the digital life. As a reader of this blog will likely have realized, a number of tools and technologies now exist that make it easy to track a life. For example, RescueTime enables you to log the time spent in different apps and sites on your computer (See: <a href="http://www.markwk.com/time-tracking-tools.html">my guide to time tracking</a>); Google’s Takeout, Twitter, Amazon’s Alexa, and Facebook let you export the bulk of your usage (like social media posts, photos, emails, voice messages, and more); read-it-later apps like Instapaper (See: my <a href="http://www.markwk.com/article-tracking-with-instapaper.html">guide to article reading tracking</a>); Apple and Android phones can track your screentime; by tweaking how you take notes you can <a href="http://www.markwk.com/track-writings.html">track your knowledge management and creative process</a>; and by owning a wearable you can track your activity, steps, sleep, heart rate, stress, and more (See: <a href="http://www.markwk.com/apple-watch-for-self-trackers.html">Self-Tracking with Apple Watch</a>).</p>
<p>In short, there is no shortage of ways to lead a digitally-mediated, data-driven and tracked life.</p>
<p>In my own life, I’m a full-time <a href="https://www.linkedin.com/in/markwkoester/">product manager</a>, and I dabble regularly in mobile and web development and data science. All of these roles put me in front of technology all the time, and I spend a large part of my non-sleeping hours on digital devices. I’m a strong believer in time tracking as an enabler of more conscious time management. To put this in perspective, according to my <a href="http://www.markwk.com/sleep-tracking-tools.html">sleep tracking</a>, I sleep about 7-8 hours a week, about one third of each day (or about 7-9 hours) goes into <a href="http://www.markwk.com/time-tracking-tools.html">computer time</a>, and another 10-16% per day (or 2-4 hours) is spent on non-computer <a href="http://www.markwk.com/2016/10/iphone-usage-tracking.html">screen time</a> either on my smartphone or tablet (including <a href="http://www.markwk.com/article-tracking-with-instapaper.html">article reading</a>, <a href="http://www.markwk.com/youtube-30-day-challenge.html">YouTube</a>, TV shows, Movies, and PDF reading). Also while it’s a bit more of a challenge to <a href="http://www.markwk.com/2016/11/book-reading-tracking.html">track my book reading</a>, even with a Kindle, I have used my <a href="http://www.markwk.com/reading-data-visualization.html">reading history to visualize what I read</a> and estimate that I read at least an hour or two per day</p>
<p>Obviously I have taken extra steps to track and make my technology and time usage more transparent, but nearly all of us today, including most millennials, live most of lives in the digital realm. So, while it is impressive that Gordon Bell took on an <a href="http://www.markwk.com/most-digitalized-life-ever.html">epic quest to digitalize everything in his life</a>), for most of us today that’s standard and par for the course.</p>
<p>And yet there are things in our lives that will remain non-digital, and that’s ok. In fact, I’d argue that it’s good to have a part of our learning, creative, and note-taking process that is analog and non-digital. A journal is a powerful tool and hand-writing is well-documented way to better retain information and build connections between ideas. In short, there is still a place for notebooks and pens today, and there are a range of techniques you can use to leverage physical paper in your creative process. This is a topic we will have to leave for another post.</p>
<p>With paper notebooks and bounded journals, there is often a moment where we want to have them digitally. Maybe you are worried that a flood or natural disaster will destroy them. Or you travel extensively and like to have the reminders for reference. Perhaps you are concerned that time will degrade them. Or you want to share a few pages with someone. Possible you desire to reuse aspects of them in an a mash-up projects. These are a just benefits and reasons why you should digitalize your notebooks. Likely you are, like many, bound to the digital world, and by making paper digital, you can make this happen.</p>
<p>While the “gold standard” for digitalization might be using an actual scanner, we all basically carry around a mini computer and digital camera that can be easily put to the same usage as an actual scanner. A number of scanner apps do a pretty incredible job of fixing the orientation, highlighting text and automating improvements.</p>
<p>In this post, we dove into the technologies and tools need to digitalize your notebooks. With a simple scanner app and a smartphone, we also did a step-by-step guide on how to go from scanning and editing journal pages to exporting to PDF and appending a table of contents. The final result is a robust digital document you can search, reference and reuse digitally. Depending on the number pages, digitalizing a notebook might only take 30 minutes to an hour or so with a scanning app.</p>
<p>Today, it’s also not unreasonable to go nearly paperless. Most of us already live that way. But some things are still nice on paper and pen. I’m a stronger believe that learning and creativity finds new avenues when expressed in analog forms. This is only possible on paper, and a journal is a great home for those sketches, ideas, and notes from the road.</p>
<p>Fortunately, it doesn’t need to one or other. Your paper journals can be made digital. You can have the best of both worlds. Paper can be made digital.</p>
<p><br /></p>
<p>Best of luck and happy digitalizing!</p>
<p><br /></p>
<p><br /></p>
<h3 id="bonus-images-scans-to-digital-text-how-to-ocr-your-scanned-docs-mac-osx">BONUS: Images Scans to Digital Text: How to OCR Your Scanned Docs (Mac OSX)</h3>
<p>In this post, we focused largely on digitalizing hand-written notebooks. But one interesting opportunity, especially if you have good handwriting is OCR technology.</p>
<p><a href="https://en.wikipedia.org/wiki/Optical_character_recognition">Optical character recognition (OCR)</a> is the conversion of scanned images into machine-readable digital text. Basically OCR software looks at the images and detects the actual letters and words. This is then superimposed on the image or document itself.</p>
<p>While a deep dive into the topic of OCR is beyond the scope of this post, OCR can be a pretty powerful addition to any quest to digitalize everything, since it makes texts searchable.</p>
<p>A number of scanners, scanning software and scanner apps provide an OCR feature, often as a paid add-on. Genius Scan includes this as one of its paid features too. Additionally there are various online, web services that can provide this too.</p>
<p>If you are on a Mac, <a href="https://ocrmypdf.readthedocs.io/en/latest/">OCRmyPDF</a> is a free, open source command line tool you can install that can process and add an OCR text layer to your PDF files. For example, when the Mueller report was first released as an image PDF, I used OCRmyPDF to convert it to text for easier reading and highlighting.</p>
<p>While the primary example I shared in this post involved handwriting and thus difficult for OCR, this approach can be applied for scans of printed pages. Also if you have clear handwriting and find the right software, you might be able to convert your paper journals into digital text docs now or one day in the future.</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}One of the most effective and powerful tools for taking notes, capturing memories and creativity in general is using a physical notebook or journal. Research has shown that writing by hand improves our retention and learning and even has benefits for our imagination too. While typing and digital photos might be the main mode most of us work in, I am increasingly convinced that in my own life about utility of a hand-written journal. Furthermore, it’s a key creative habit that all creatives from writers and artists to programmers and data scientists should consider using.Gordon Bell and The Epic Quest to Digitalize Everything2019-10-02T00:00:00-07:002019-10-02T00:00:00-07:00http://www.markwk.com/most-digitalized-life<p>Gordon Bell might rightly claim the prize for one of the most <em>digitalized</em> life ever. Born in 1934 and coming of age after World War II, Bell witnessed and spurred on a societal shift from paper to digital, from paper print-outs and letters to digital documents, PDFs, email, and all-things digital. Bell was an American electrical engineer and innovator in the early days of computers. He worked on several ground-breaking technology products. But he is arguably most well-known for his quest to digitalize his entire life and develop products to take advantage of such a life.</p>
<p>Bell’s project, which was known as <strong>MyLifeBits</strong>, was part research project at Microsoft, part experiment in comprehensive lifelogging. In a sense, Bell and his team of engineers were taking up Vannevar Bush’s vision from 1945 for how technology might augment and help us our memories and attempting to build it. Bush, who was both a scientist himself and a key government official guiding scientific policy and research in post-ware America, laid out what many consider to be a prophetic vision for much of the technological creations that came to be in the decades ahead.</p>
<p>At the time, Bush was lamenting the struggle to keep abreast of so much research and publications and how to productively take notes, and use that knowledge in his own thinking and writing. He felt limited by piles of paper. So, he proposed several ways to overcome these limitations and developed a new paradigm of working and remembering he called the <strong>Memex</strong>. While Bush wasn’t able to realize much of his vision during his life, during Gordon Bell’s own lifetime various technological changes had made it possible to take up this quest again in the late 1990s and early 2000s.</p>
<p>The aim of MyLifeBits was to digitalize as much of a single human life as technically possible at the time. It also explored, researched, and built software solutions in support of this “lifetime store of everything.” The end result was two-fold: an assortment of software and integration solutions and Bell’s own lifestore or digital archive.</p>
<p>Admittedly, this <strong>quest to digitalize a life</strong> is an odd one. It sits at the intersection of a remarkable computational and historical change. For Bell there was a clear boundary between his early work and personal life in papers and the later one on computers, digital cameras, GPS, and smartphones.</p>
<p>Today most of our lives come already packaged in digital. We are more apt to get an email than a letter, more commonly read digital words than printed ones, and are more likely than ever to engage in large portions of our lives through digital tools and online spaces rather physical ones. And yet Bell’s somewhat quixotic quest to digitalize a life offers us an illuminating story on how one might live with and through technology both today and in the future. It gives us a number of lessons and even shows us some of the limits to such a technology-infused lifestyle.</p>
<p>Let’s dive into the tale of one engineer’s quest to go paperless!</p>
<p><br /></p>
<p><em>NOTE: This post covers the historical and visionary aspects of personal digitalization. See a <a href="http://www.markwk.com/digitalize-paper-journals.html">Guide to Digitalizing Your Journals with a Scanner App</a> for a practical guide on how to digitalize your own papers and journals.</em></p>
<!--more-->
<h2 id="a-short-history-of-the-epic-quest-to-digitalize-everything">A Short History of the Epic Quest to Digitalize Everything</h2>
<p><img src="/generated/images/2019-resources/head-camera-memex-800-bb4873d54.jpg" srcset="/generated/images/2019-resources/head-camera-memex-400-bb4873d54.jpg 400w, /generated/images/2019-resources/head-camera-memex-600-bb4873d54.jpg 600w, /generated/images/2019-resources/head-camera-memex-800-bb4873d54.jpg 800w, /generated/images/2019-resources/head-camera-memex-1000-bb4873d54.jpg 1000w" /></p>
<p><small>Source: Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101-108.</small></p>
<h3 id="bushs-memex-a-1945-technological-prophesy">Bush’s Memex: A 1945 Technological Prophesy</h3>
<p>Vannevar Bush’s 1945 article “As We May Think” remains a key document in the history of computing. It foretold many aspects of computing we take for granted today yet still rings oddly contemporary in his hope to make reading and knowledge management more manageable.</p>
<p>(NOTE: <a href="http://www.markwk.com/plain-text-life.html">The Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files</a> is an in-depth write-up I did on knowledge management using plaintext file notes. Check it out for implementing a mesh of notes and knowledge management towards creative output.)</p>
<p><img src="/generated/images/2019-resources/memex-illustration-450-50d003d69.jpg" srcset="/generated/images/2019-resources/memex-illustration-400-50d003d69.jpg 400w, /generated/images/2019-resources/memex-illustration-450-50d003d69.jpg 450w" /></p>
<p>Bush, who at the time was the head of of the Office of Scientific Research and Development, lamented the difficulty of keeping up with the mountain scientific research and frailty of remembering and navigating that information over time. He proposed several technical solutions, including:</p>
<ul>
<li>Cyclops Camera - a forehead worn camera that could record anything you see and want to capture.</li>
<li>Microfilm</li>
<li>Vodocoder - a machine that could do speech-to-text transcription</li>
<li>Thinking Machine</li>
<li>“an aid to memory. Like the brain, Memex would file material by association. Press a key and it would run through a ‘trail’ of facts” (p. 113)</li>
</ul>
<p>Most famously he came up with a fundamental paradigm for how to organize and link data together, which he called Memex.</p>
<p>Bush’s Memex, which stood for memory extender, was a microfilm-based “device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.” (p 121). Basically he was describing a knowledge management system. At the heart his insight is the idea of <strong>linking together of information by association</strong>. Today we would call these hypertext links.</p>
<p>As Bush foresaw in 1945:</p>
<blockquote>
<p>“A record if it is to be useful … must be continuously extended, it must be stored, and above all it must be consulted…The difficulty seems to be, not so much that we publish unduly … but rather that publication has been extended far beyond our present ability to make real use of the record”</p>
</blockquote>
<p>Much like how the brain links ideas together through association, Bush imagined a system “ready made with a mesh of associative trails running through them.” He envisioned complete storage, full-text search, text and audio annotations, and hyperlinks. Furthermore this way of connecting was not just for organization, the memex through these associated trails was intended as a way to amplify that knowledge. It wasn’t just knowledge capture and storage; it was designed as extension of our creativity, our learning and thinking too.</p>
<p>Unfortunately, for Bush, the technology available at that time of his life never allowed for Memex to see the light of day. It remained a futuristic vision. By contrast, by the time Gordon Bell was dealing with similar challenges and conceiving his own e-memory project, various technologies had come together to make this type of quest not only possible but increasingly feasible using cheap consumer technologies at hand.</p>
<h3 id="gordon-bells-quest-to-digitalize-everything">Gordon Bell’s Quest to Digitalize Everything</h3>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/d/d2/Gordon_Bell.jpg" alt="" /></p>
<p><small>Gordon Bell. Source: Wikipedia.</small></p>
<blockquote>
<p>The MyLifeBits project began as an effort to digitize the books Gordon has written, and it has turned into a pioneering endeavor to record and preserve in digital form everything he sees, hears, learns, and experiences. (p 8)</p>
</blockquote>
<p>Going under the singular title of MyLifeBits, the project itself was multifaceted. Along with his long-time collaborators and fellow software engineers, Jim Gemmell and Roger Lueder, Bell and company’s initial efforts were two-pronged: First, they wanted to record and digitalize everything from Bell’s paper archive from the past. Second, they wanted to assemble tracking tools that allowed them to record everything in the present and going forward, including emails, biometric data and even a stream of photos taking from a first-person perspective. Eventually though digitalizing and tracking were key inputs, they realized capturing wasn’t going to be enough; they need a way to intelligent use all this digital information.</p>
<p>Bell, having started work in a pre-digital era and seen the rise of personal computing, felt a pull to bring his file cabinets of papers and boxes of photos into the digital age too. So, the first stage of their project was aimed at scanning all of his papers.</p>
<p>Using a commercial scanner and a research assistant, they converted this paper archive from the past into a digital one. In total, they scanned virtually all of his:</p>
<ul>
<li>Books written (and read when possible)</li>
<li>Personal documents (correspondence including memos and email,
bills, legal documents, papers written, …)</li>
<li>Photos</li>
<li>Posters, paintings, photo of things (artifacts, …medals, plaques)</li>
<li>Home movies and videos</li>
<li>CD collection</li>
</ul>
<p>They would eventually hire a team to scan and annotate every scape of paper in Bell’s life, a process that would take a number of years. His paper filing cabinets emptied as he digitalized everything, filling the storage capacity of external drives at the time.</p>
<h3 id="mylifebitss-vision-capture-everything-bring-it-together">MyLifeBits’s Vision: Capture Everything, Bring It Together</h3>
<p>Beyond just digitalizing everything he had on paper, Bell and company also strove to proactively digitalize his life going forward.</p>
<p>In what is known as “lifelogging,” the stated goal is to digitize everything that happens to someone. This includes recording images, video, and audio from everything around us. As Wired Editor Kevin Kelly puts it, “Lifestreaming can be thought of as an active, conscious tracking.” It is one of the most extreme forms of self-tracking.</p>
<p>Working in the early 2000s, for Bell to engage in lifelogging, he was forced to build many of his own tools to keep a log or record his life. For example, using some custom software, they recorded his computer activities, like websites visited and the text on those sites; they experimented with GPS trackers to monitor geospatial movements; and, in the spirit of Bush’s head-mounted version, they even developed a camera to take first-person perspective photos as Bell went about his life. In fact, the camera they developed eventually became the Microsoft product, SenseCam.</p>
<p>Ironically, while the term “lifelogging” has lost popularity, today various commercial products, apps and tools now exist to make tracking your life easier than ever, and it can largely be done passively using a smartphone, a wearable, and a few self-tracking apps.</p>
<p>(NOTE: I’ve written extensively on this topic in series on <a href="http://www.markwk.com/category/track-everything/">how to track a life</a>. For example, among other things, <a href="http://www.markwk.com/time-tracking-tools.html">RescueTime</a> can be used to record your computer usage, <a href="http://www.markwk.com/article-tracking-with-instapaper.html">Instapaper</a> can log your article reading, most phones can tell you your screentime, and a wearable like <a href="http://www.markwk.com/apple-watch-for-self-trackers.html">Apple Watch</a> can help you know your <a href="http://www.markwk.com/2016/09/counting-steps.html">steps</a>, movements, <a href="http://www.markwk.com/how-to-track-workouts-fitness.html">workouts</a> and even <a href="http://www.markwk.com/sleep-tracking-tools.html">sleep</a>. You can even use an app like <a href="www.photostats.io">PhotoStats.io</a> to know more about when, where and what you take pictures of on your phone.)</p>
<p>Eventually though the MyLifeBits team realized that digitalizing and lifelogging were only part of the challenge; they needed a way to make this collected information useful and usable. Simply having all of this information in an unorganized or unstructured format created more problems.</p>
<p>In a series of research papers culminating in their 2010 book, <em>Your life, uploaded: The digital way to better memory, health, and productivity</em>, Bell and Gemmell’s focus went to the actual system, MyLifeBit, that would store, link and annotate all this information together.</p>
<p>Here is an early sketch on the MyLifeBits store:</p>
<p><img src="/generated/images/2019-resources/mylifebits-lifestorage-schematic-800-82d0a7360.jpg" srcset="/generated/images/2019-resources/mylifebits-lifestorage-schematic-400-82d0a7360.jpg 400w, /generated/images/2019-resources/mylifebits-lifestorage-schematic-600-82d0a7360.jpg 600w, /generated/images/2019-resources/mylifebits-lifestorage-schematic-800-82d0a7360.jpg 800w, /generated/images/2019-resources/mylifebits-lifestorage-schematic-1000-82d0a7360.jpg 1000w" /></p>
<p>At the core of this vision is a database. This is a singular place where everything gets linked together and/or stored. It includes various digital files, photos and videos already there. Additionally, going into this database are various lifelogging or tracking streams. This includes data like voice annotations, internet usage, TV watching, email, radio and many, many more. Everything would have meta-data allowing you to contextual the time of creation and where it happened.</p>
<p>You might think about this software as personal internet or a form of e-memory where everythinig is linked together. This allows you to search your history on every digital and digitalized aspect of a life.</p>
<h3 id="sidebar-the-most-informative-media-conversational-audio">SIDEBAR: The Most Informative Media: Conversational Audio</h3>
<p>In one of the most interesting and insightful asides from the entire project, Bell remarks that amongst all the lifelogging and tracking, the most informative media he found was audio from conversations.</p>
<p>While it might seem like visual information and photos are the best thing to track and learn from, for him our conversations are one of the most powerful lifelogging data sources we have. In fact, if he could only lifelog one thing, it would be his conversations. He preferred to have the audio or even better text from his daily conversations rather than just a collection of photos or visual data.</p>
<p>Sadly, this is the one feature that quantified self technology has yet to provide, though tools like Alexa, Google Assistant and Siri might one day provide this.</p>
<h3 id="total-recalls-three-streams-of-data-and-technologies-that-make-it-possible">Total Recall’s Three Streams of Data and Technologies that Make It possible</h3>
<p>The MyLifeBits project offered us a new paradigm for living they called “Total Recall.” Essentially instead of being limited to our biological memory, all of this lifelogging enables e-memory, a digitalization and extension of our “wetware” brains through digital storage and retrieval.</p>
<p>According to the team, this “Total Recall revolution is being birthed on the strength of a few key devices: cell phones, digital cameras, and pocket-size GPS units.” (p 186). Much like the Quantified Self Movement, they define three streams of technologies that make this vision possible: recording, storage, and sophisticated recall. It’s worth quoting this in full:</p>
<blockquote>
<p>Three streams of technology are coming together to make the world of Total Recall a reality.</p>
<p>First, and perhaps most important, we are recording more and more of our lives digitally without even trying. Digital cameras, e-mail, cell phones, and personal digital assistants (PDAs) are the vanguard of technology that is generating an explosion of digital records of our daily lives. Digital sensing and recording will become ubiquitous.</p>
<p>Second, this mountain of new personal digital records can now be stored more cheaply than can easily be imagined—for about two hundred dollars you can own enough memory to store everything you read, everything you hear, and ten pictures a day for your whole life.</p>
<p>Third, technologies enabling you to search, analyze, and present all kinds of reports from such large mountains of data are being developed, with astonishing results. Google will by no means be the last extraordinarily successful company to be built on new search technologies. So, we live in a world with more digital memories, more space to store them, and better and better technology to recollect them. The world of Total Recall is inevitable for these three reasons.
(p 11)</p>
</blockquote>
<p>In summary, various <strong>tracking technology</strong> makes capturing easier and more ubiquitous from our devices, from our bodies via wearables, and from our environment via remote sensors; <strong>computer and cloud storage</strong> is also cheaper than ever, meaning it isn’t cost prohibitive to never throw away anything digital; and since we have a nearly omnipotent <strong>ability to index and search all of this data</strong>, personal and universal, we can recall, pull out and put together the information and data.</p>
<p>In the same spirit as Kevin Kelly’s wonderful tech-future vision and book, <em>The Inevitable</em>, Bell believes e-memory and total recall are here to stay and will be a dominant aspect of the technology landscape going forward.</p>
<h3 id="the-mylifebits-prototype-making-life-data-usable-and-useful">The MyLifeBits Prototype: Making Life Data Usable and Useful:</h3>
<p>One key feature of MyLifeBits was to be an e-memory tool that could augment and extend our biological memory. As they put it:</p>
<blockquote>
<p>Software will allow you to sort and sift through your digital memories to uncover patterns in your life you could never have gleaned with your unaided brain. Your work habits, your leisure habits, and your spending habits; your emotional response patterns in various situations and around certain people; the numerous subtle factors that affect your mental well-being and your physical health; and just about anything else you care to know about yourself can be chronicled, condensed, cross-correlated, and plotted out for you in useful and illuminating ways. (p 12)</p>
</blockquote>
<p>So, after years of efforts digitalizing one man’s paper and photo archive, developing tools to lifelog, and integrating that data together, a major challenge remained for Bell and the team at MyLifeBits:</p>
<p>What to do with all of this data? Could they make it useful and usable? What sort of product or service it might be?</p>
<p>It was time to take this lifelogging and digital archive and make it usable and useful:</p>
<blockquote>
<p>The challenge now is to make the stored material useful. MyLifeBits implements a number of features originally proposed by Vannevar Bush, such as links, annotations, and full text search. This demonstration highlights the importance of augmenting these ideas with typed links and database features. Visualizations, complex queries, and pivots enable the user to find or browse many items that would otherwise remain unused.</p>
</blockquote>
<p>It was time to build a prototype.</p>
<p>Here are some early prototypes of the product:</p>
<p><img src="/generated/images/2019-resources/mylifebits-ui-prototype-800-dd93f6e3b.jpg" srcset="/generated/images/2019-resources/mylifebits-ui-prototype-400-dd93f6e3b.jpg 400w, /generated/images/2019-resources/mylifebits-ui-prototype-600-dd93f6e3b.jpg 600w, /generated/images/2019-resources/mylifebits-ui-prototype-800-dd93f6e3b.jpg 800w, /generated/images/2019-resources/mylifebits-ui-prototype-1000-dd93f6e3b.jpg 1000w" /></p>
<p>Underlying this product was a database of annotated and linked metadata. It wasn’t arranged into folders, but interconnected collections of data. To use it, you could do a search and use filters to refine it by periods of time and various other facets.</p>
<p>Additionally, various charts and data visualizations could be created, like which websites were visited during certain periods:</p>
<p><img src="/generated/images/2019-resources/bell-tracked-life-sites-visited-700-8c0883320.jpg" srcset="/generated/images/2019-resources/bell-tracked-life-sites-visited-400-8c0883320.jpg 400w, /generated/images/2019-resources/bell-tracked-life-sites-visited-600-8c0883320.jpg 600w, /generated/images/2019-resources/bell-tracked-life-sites-visited-700-8c0883320.jpg 700w" /></p>
<p>These graphs show that once you have a collection of life data, you have an opportunity to see trends and display patterns toon. Bell’s charts are also reminiscent of my own efforts at visualizing tracking data using Tableau or with <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>. For example, see my recent <a href="http://www.markwk.com/2019/01/year-in-data.html">A Year in Data</a>.</p>
<p>Based on these prototypes and examples, MyLifeBits represented an epic product to capture everything, bring it together, make it searchable, and provide a data-driven tool for a well-tracked life. For anyone hoping to digitalize their life, this was the dream product many of us had been hoping for.</p>
<p>So, what happened? Where can I get a version of MyLifeBits?</p>
<h3 id="what-happened-to-the-mylifebitss-product">What Happened to the MyLifeBits’s Product?</h3>
<p>Unfortunately, at the time of writing this post, it appears that neither the software for MyLifeBit nor a public version of Bell’s archive ever saw the light of day. One of the products created in conjunction, SenseCam, which was a wearable lifelogging camera, is also no longer available, though alternative wearable lifelogging cameras are still around.</p>
<p>It’s been reported that aspects of the MyLifeBits’s technology was applied to Microsoft’s desktop and cloud search tools. While Bell might have succeeded in his epic, albeit quixotic, quest to go paperless and digitalize everything in his life, it should be admitted that much of the hype and promise of a comprehensive e-memory remains unfulfilled, especially if judged by the fact that no user-facing product ever got released.</p>
<p>What happened to the MyLifeBits’s product? Was it too hard technically? Did the system require too much manual work to integrate and annotate information? Was usability too tedious? Was it just a service no one really wanted? What the heck happened?</p>
<p>For one thing, I think the vision of a comprehensive tool may not be what people wanted or needed. Consider this: Google already makes informational searches seamless and magical. File search on your computer or in the cloud has become robust and makes finding what you need relatively easy. Various note-taking tools now help you store and find your notes too. Similarly, Apple, Flicker, Google and even my own photo app, <a href="www.photostats.io">PhotoStats.io</a> have all significantly improved aggregating and searching our personal photos. So, while all of these services are fragmented and siloed and none really bring it all together or make it data-driven, several major services provide incredible indexing and searching functions for public information, files, notes and photos.</p>
<p>There remains one area that no service has really solved and that’s aggregating and using various self-tracking and lifelogging data. While it’s easier than ever to track a life, it’s still a challenge to bring this data together and make it useable and useful. This means that if MyLifeBits’s software was to find a meaningful and impactful use case, I’d argue that it would most effecive in this situation of aggregating and augmenting our personal tracking data towards the data-driven life.</p>
<p>Admittedly, even though a software product never emerged, Bell has written a few books that act as a surrogate for the vision he proposed. Much like Bush, he offers a mix of technical predictions and prophetic mash-ups for how technology and a way of living might emerge. I don’t necessarily subscribe to every claim, but it does provide good fodder for futuristic thinking.</p>
<p>In my own work, while I didn’t intentionally plan it, my open source, data aggregation and visualization tool <a href="https://github.com/markwk/qs_ledger">QS Ledger</a> is also a step in the same direction as MyLifeBits and Bell’s vision. For example, QS Ledger allows you to pull your data together from 20+ sources and visualize it. It allows you to contextually see your data and to tell data stories. You can use it to ask and answer questions a whole host of questions on your life using your life tracking data.</p>
<p>Admittedly, much work remains on that project and others if we hope to translate all of this tracking into products that allow us to have and augment a data-driven life.</p>
<h2 id="conclusion-a-quixotic-quest-to-digitalize-everything">Conclusion: A Quixotic Quest to Digitalize Everything?</h2>
<p>Over 75 years ago, Vannevar Bush’s manifesto, “As We May Think,” offered a futuristic vision called Memex for capturing and extending our minds through technology. Ironically he foresaw this as being done using microfilm. Computing pioneer Gordon Bell took up this same vision and attempted to fabricate a version of this using computers, sensors and other innovations. Through a series of projects under the umbrella of MyLifeBits, Bell digitalized all of his own papers and photos, developed technologies for self-tracking and lifelogging, and built software to make it searchable. While MyLifeBits software failed to materialize, many parts of Bell’s vision and implementation have to come to be in dozens of technologies and companies that help us track our lives, find digital information, and lead a data-driven life.</p>
<p>In the book, <em>Your life, uploaded: The digital way to better memory, health, and productivity</em>, Bell and his co-author and lifelong collaborator Jim Gemmell summarize their project thus:</p>
<blockquote>
<p>Our goals were twofold:</p>
<ol>
<li>To create software for lifelogging, and the subsequent recall and usage of one’s e-memories. We wanted software to record a diverse array of information about one’s life and activities, from a variety of sources and devices, and to do so as easily, as unobtrusively, and as automatically as possible. The software would have to give people powerful tools for searching, organizing, annotating, and pattern-mining their ultimately huge e-memories.</li>
<li>To identify the benefits, drawbacks, technical issues, sticking points, and usability of Total Recall in real life. We wanted to try it out (as much as we could) and see what it was like.
(p 40)</li>
</ol>
</blockquote>
<p>Many of us already live, at least to some extent, in a world of total recall. When our biological memory slips, tech can help us. We can check our social media feed for life events, we search our calendar for past trips and meetings, we can scan our expansive email, photo libraries, and chat logs for play-by-play, and even look at biometric data on movement, sleep and heart rate to visualize our hourly health journey.</p>
<p>Looking at Bell’s project holistically. I’m struck by two things:</p>
<p>First, Bell’s world existed at the borderland between the materiality of a paper and printed past and today’s already digitally-infused worldview. He spent several decades developing computer technologies while also seeing his paper file cabinets grow. For this reason, digitalization was an obvious goal for him and many others like my parents who lived through the transition from physical papers and photos to cloud storage and social media. Unfortunately, this kind of half-digital life has become increasingly rare. We largely no longer need to digitalize, since much of life already comes packaged in digital. Many of us spend most of our non-sleep hours looking at a screen or carrying around a mini, ever-present tracking device.</p>
<p>For this reason, much of Bell’s discussion of digitalization rings dated even a few years after publication. For many of us today, paperless and digital is the norm, rather than the exception. There are still areas and parts of our lives that could use digitalization, especially how to digitalize a journal or hand-written letter. This is a topic I’ll take up in Part 2 on the practical side of going paperless using scanning apps.</p>
<p>The second thing that strikes me about Bell’s epic quest to digitalize everything is how individualistic it comes off as on one level. You see elements of this in how his children comment on the project too. It’s not something for everyone. And for as much as Bell couches his work as a step towards technology and software that will help humanity digitalize and lead a data-driven life, the failure to deliver any products makes you wonder if about his underlying mission and obcession.</p>
<p>For example, Bell talks about the role self-tracking data might play in learning, health and more. But did Bell really care about helping society evolve towards a useful ubiquity of lifelogging and personal data products? Or was it more about a single human’s drive to create an artifact and memorial to his own temporary existence? Was it epic or quixotic?</p>
<p>Either way, it appears Bell did largely go paperless, and he can rightly be called a trailblazer in the lifelogging, quantified self and data-driven life. Bell’s efforts and projects are indeed a unique curiosity and tale in the history of humanity and of computing as he strove to become the most <em>digitalized</em> life ever.</p>
<p><br /></p>
<p><br /></p>
<h5 id="sources">Sources</h5>
<ul>
<li>Bush, V. (1945). As we may think. The Atlantic Monthly, 176(1), 101-108.</li>
<li>Bell, G., Gemmell, J., & Lueder, R. (2004). Challenges in Using Lifetime Personal Information Stores based on MyLifeBits. Proceedings from Alpbach Forum.</li>
<li>Bell, G., & Gemmell, J. (2010). Your life, uploaded: The digital way to better memory, health, and productivity. Penguin.</li>
<li>Bell, G. (2014). Supercomputers: The Amazing Race (A History of Supercomputing, 1960-2020).</li>
<li>Cherry, S. (2005). Gordon Bell, A Life Digitalized. IEEE Spectrum.</li>
<li>Gemmell, J., Lueder, R., & Bell, G. (2003). The MyLifeBits lifetime store.</li>
<li>Gemmell, J., Bell, G., Lueder, R., & Drucker, S. (2002). MyLifeBits: fulfilling the Memex vision. Communications of the ACM.</li>
<li>Gemmell, J., Bell, G., & Lueder, R. (2006). MyLifeBits: a personal database for everything. Communications of the ACM (CACM).</li>
<li>Selke, S. (2016). Lifelogging: Digital self-tracking and Lifelogging - between disruptive technology and cultural transformation (1st ed. 2016 ed.). Springer VS.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Gordon Bell might rightly claim the prize for one of the most digitalized life ever. Born in 1934 and coming of age after World War II, Bell witnessed and spurred on a societal shift from paper to digital, from paper print-outs and letters to digital documents, PDFs, email, and all-things digital. Bell was an American electrical engineer and innovator in the early days of computers. He worked on several ground-breaking technology products. But he is arguably most well-known for his quest to digitalize his entire life and develop products to take advantage of such a life.Instapaper: Empowering How I Read Articles with Highlights and Tracking2019-09-09T00:00:00-07:002019-09-09T00:00:00-07:00http://www.markwk.com/instapaper-tracker<p>Are you facing a never ending pile of articles to read? Have you ever remembered this amazing post you recently read but forgotten the title or URL? Or are you just hoping to augment and improve how you read and learn stuff online using technology?</p>
<p>One empowering approach to reading and learning in the internet age are read-it-later apps. My tool of choice is Instapaper, and with a few additional tweaks, as I’ll go into below, it can be even better.</p>
<p>A read-it-later app enables you to save articles from the web and (surprise, surprise) read them later. They generally give you a better reading experience without all of the ads and distractions too. They are largely intended as a way to capture stuff that comes up while working on one thing, avoid distraction, and read it when it is more convenient.</p>
<p>The three most popular approaches today are Pocket, Instapaper and Evernote. Each of these lets you save articles and then read them offline on a mobile device or computer using their app.</p>
<p>At their most basic, each make article reading experience better and less distracted. Each can also be leveraged to build an external, digital memory of the things you read. You can look back in time and know what you read and when. You can create even a <a href="http://www.markwk.com/2019/01/year-in-data.html">year in reading</a> as I did to see trends. Moreover, when your biological memory slips, you can simply search your archive of saved articles to find the reference or quote that stood out.</p>
<p>As I recount in <a href="http://www.markwk.com/2016/11/tracking-article-reading.html">How to Track the Articles You Read and Augment Your Digital Memory</a>, the two tools I previously used were Evernote and Pocket. Both are good options, but over the last year or so, I’ve become a real fan of <strong>Instapaper</strong>.</p>
<p>On the surface, Instapaper is extremely similar to the slightly more well-known and popular, Pocket. Not only does Instapaper provide a beautiful and clean reading experience, it gives some empowering extra features, especially if you depend on internet research for your work or studies.</p>
<p>The two features that most stand out for me are highlights and tracking:</p>
<ul>
<li><strong>Highlights</strong> enable me to mark up important passages as I read them. I use a similar feature on Kindle and for PDFs too. While highlights don’t automatically create memories and learning and in fact we shouldn’t forget that highlighting and re-reading are two of the least effective <a href="http://www.markwk.com/science-of-learning.html">learning techniques</a> (Dunlosky, 2013), they are a good starting point, since instead of re-reading an entire article, you can just extract the key parts you highlighted. What I often do when I’m researching a new topic is during my initial search, I first save various articles into Instapaper. After that, I read those topic-focused articles in the app and highlight accordingly. Then once I’m ready, I export all of the highlights and use them to summarize in my own words. This is part of my knowledge cycle and my general <a href="http://www.markwk.com/plain-text-life.html">note-taking technique</a>.</li>
<li>The other thing I love about Instapaper is its <strong>tracking capability</strong>. Unfortunately, Instapaper doesn’t provide any internal way to track or visualize your reading history just yet. Fortunately, it is easy to set it up your own tracking and visualizations. By either accessing its API with some code or using its integration with IFTTT, you can extract your entire reading history, including article read, articles liked and all of your highlights too.</li>
</ul>
<p>Ultimately, this combination of a beautiful reading experience, highlights and tracking have made Instapaper my go-to app for reading articles now.</p>
<p>In this post, I want to explain what is Instapaper and how to set it up for tracking and knowledge management. I’ll share how to export your highlights locally, walkthrough a bit of code for data analysis, and show how you can use Google Sheets for cool and simple data visualizations too. At the end, in my conclusion, I share my dream for the next, most futuristic read-it-later apps.</p>
<p>Once we are done, I hope that you too will be ready deploy your own empowering article reading and learning system with Instapaper!</p>
<!--more-->
<h2 id="instapaper-an-empowering-read-it-later-tool">Instapaper: An Empowering Read-It-Later Tool</h2>
<p>Setting up Instapaper is pretty straightforward. First create an account. Next, on your phone or tablet download and install the app, and on your browser, install the <a href="">Instapaper browser extension</a>. Instapaper has a good guide on all the ways to <a href="https://www.instapaper.com/save">save links</a>, including email, RSS and many other tools.</p>
<p>After you are setup, all you need to do is start saving links, and Instapaper will do the rest of the work. As links are saved, Instapaper will strip out the ads and give you a nice looking and consistent way to read. Whether or not I’m technically reading it later, I find that the Instapaper app and web version are both a much better manner to read online.</p>
<p>The main reason you want to the app and browser extension is to make it faster and almost thoughtless to save articles to read later. The objective is to build an automatic habit or routine when working on the web. Instead of getting distracted reading something new or interest in the moment, avoid the distraction and save it for later. By focusing in the moment and having good stuff to read later, you get the best of both worlds.</p>
<p>Obviously you can read articles directly as they come up, but the other reason to save articles is that it gives you a nearly complete record of the key articles you read throughout the day. This is part of Instapaper’s capacity to help you track your media consumption. Personally, anything I’m going to spend the time reading online gets saved into Instapaper. While I may not always read it in the app, I do my best to save and archive what I read once a day. This means nearly every important article I read online is captured and tracked. That way I have a decent record and count of my reading habits. We will look more at the tracking part shortly.</p>
<p>The other feature I love about Instapaper are highlights. Highlights are a way of marking up key passages and parts of something you are reading. All you have to do is select a passage of text while reading in the app or in the web version and select highlight. Highlight saved!</p>
<p>I use highlights in regular book reading on my Kindle and on PDFs too. Instapaper brings this ability to my article reading, so that as I read a part of an article that I find important or noteworthy, I highlight it. All of my highlights then get saved for me to use and reference later.</p>
<p>A number of apps today make it possible “highlight” the web, even Pocket now provides this option. I have researched and tried some of these tools, like Liner, but never found the experience very good across devices and systems. The big problem with Pocket’s highlight feature is there is no good way to get them out! You can highlight but you can only export them one by one. No integration with IFTTT and there isn’t even a way to get them from the API. Sad.</p>
<p>(<em>NOTE: Instapaper only provides a limited number of highlights for the free version. So, in order to take advantage of unlimited highlights, you’ll likely need a premium account.</em>)</p>
<p>The other nice thing about Instapaper is Search. If you are anything like me, then there is a good chance you regularly come across a number of interesting things to read but rarely have enough time to read it all. Moreover, finding good material doesn’t always mean now is the right time to read that material or topic. That’s where search in Instapaper can be super useful.</p>
<p>Basically by saving interesting material over time into Instapaper, you have an archive you can access later when you want to explore that topic. Instead of directly searching that topic, you can first search in Instapaper to see you already saved something on that topic. This makes it easy to find targeted material that you already noticed (and maybe even read and forgot about it), and then use them as a starting point for reading and learning more on a topic.</p>
<p>These are the core features I use with Instapaper. Worth mentioning if you are curious on other features are:</p>
<ul>
<li>Folders: These are specific areas that allow you to save articles to for later and provide more organization.</li>
<li>Browse: This is Instapaper aggregator of great material found online. It is mostly a best of space. You can apparently link to twitter and facebook accounts to see and share what others are reading too.</li>
<li>Speed Reading: This is a premium feature that will display articles word by word in high speed and can increase how fast you read. I’ve tried similar approaches in the past but never been a huge fan of reading like this.</li>
</ul>
<p>Now that we’ve looked at how to install and use Instapaper, it’s time to dig into a customized setup that prioritizes tracking and learning.</p>
<h2 id="tracking-your-article-reading-and-highlights-with-ifttt-and-qs-ledger">Tracking Your Article Reading and Highlights with IFTTT and QS Ledger</h2>
<p>Instapaper is great by itself, but it’s even more empowering when we customize and add a few more integrations to help us with tracking, visualizing and using what we read and highlight.</p>
<p>At present Instapaper doesn’t provide a direct way to visualize your reading trends in the app or on the website. This is unfortunate and would be a great feature to see sometime in the future. For example, it would be interesting to see my reading trends and habits and maybe clustered topics over time.</p>
<p>Fortunately, Instapaper does provide two excellent options to get your reading data for tracking and analysis purposes. Let’s look at these two options one by one:</p>
<h3 id="automated-article-tracking-with-ifttt">Automated Article Tracking with IFTTT</h3>
<p>IFTTT stands for “If This, Then That…” and is one of the most popular automation and integration tools available today. At its core, IFTTT enables you to connect two online services so they work together. For example, in the morning you might get a custom weather notification that it is going to rain today or be colder. You might create a workflow that posts to social media each time you publish a new blog post. Or save an instagram photo to a particular cloud service.</p>
<p>IFTTT has hundreds of integrations making it possible to have one app work and share information with another. For self-trackers and data freaks, it’s one of the easiest ways to get data out of one tool and into another. The most common example is take tracking data from something like Strava, Fitbit, Runkeeper and many others and log your data into a Google Sheet. So every time you complete a workout, the data is saved to a spreadsheet. I also use this same approach to log workouts into my calendar too.</p>
<p>Instapaper and IFTTT have a really great integration that makes it possible to keep a log of every article you save, read and like. You can also use it to save your highlights too.</p>
<p>Setting up and using IFTTT is really easy. First thing you need to do is create an account. After that, you need to connect different services. Finally, you need to create an integration workflow or recipe which IFTTT calls “Applets.” An Applet requires a trigger (the “If”) and a reaction (the “That”). For example, the trigger might be rainy conditions and the reaction might be to send you a text message or, if you wanted to track it, you could log it to a document.</p>
<p>For Instapaper, all you need to do is search for that service and connect it to IFTTT. After that, you’ll need to connect another service. The easiest and most reliable I’ve found is Google Sheets.</p>
<p>After you’ve connected both services, it’s time to <a href="https://ifttt.com/create">create an applet</a>. First select “This” and search for Instapaper and then choose one of its triggers. In this case, we will use “New archived item” which is when you finish reading an article. Second, select “That” and search for Google Sheets and then choose the option, “Add row to spreadsheet.” Now all you need to do is customize the title and spreadsheet location.</p>
<p>Here is what my final recipe looks like:</p>
<p><img src="/generated/images/2019-resources/ifttt-article-tracking-instapaper-800-66dea3e36.jpg" srcset="/generated/images/2019-resources/ifttt-article-tracking-instapaper-400-66dea3e36.jpg 400w, /generated/images/2019-resources/ifttt-article-tracking-instapaper-600-66dea3e36.jpg 600w, /generated/images/2019-resources/ifttt-article-tracking-instapaper-800-66dea3e36.jpg 800w" /></p>
<p>Now that our applet is ready, each time you read and archive something in Instapaper, you’ll have a record of it in Google Sheets. I recommend setting up similar workflows for Likes and Highlights.</p>
<h3 id="accessing-your-instapaper-article-history-using-qs-ledger">Accessing Your Instapaper Article History using QS Ledger</h3>
<p>IFTTT works great, but what if you don’t want to use a third-party tool or you already have saved articles in Instapaper?</p>
<p>In that case, the best approach is pulling your history directly from the Instapaper API. Instapaper’s API is quite well documented and easy to use. In this example we are going to show how to use Python and QS Ledger to get your reading history.</p>
<p><a href="https://github.com/markwk/qs_ledger">QS Ledger</a> (short for Quantified Self Ledger) is an assortment tools for data collection and data visualization. It’s written and developed in python code using Jupyter Notebooks and currently aggregates and processes data from nearly 20 sources, including Fitbit, Apple Health, Google, RescueTime and many others. It can provide a starting point to analyze and <a href="http://www.markwk.com/2019/01/year-in-data.html">visualize a life in personal data</a>. It can be used to understand how you sleep, your exercise behavior, or how much time you spend on the computer or listening to music. It can also be used to pull data on what articles you read from both Pocket and Instapaper. Each project and integration includes a data downloader and data analysis notebook.</p>
<p>A tutorial on Python is outside the scope of this post. To get started, I recommend downloading and using the <a href="https://www.anaconda.com/download/#macos">Anaconda Distribution</a>. Then run Jupyter notebook and open up <a href="https://github.com/markwk/qs_ledger/blob/master/instapaper/instapaper_downloader.ipynb">instapaper/instapaper_downloader.ipynb</a>. This code will walk you through the steps need to use the Instapaper API and get your data!</p>
<p>After you’ve got your data, I recommend you checkout the <a href="https://github.com/markwk/qs_ledger/blob/master/instapaper/instapaper_data_analysis.ipynb">Instapaper data analysis notebook</a> which provides a good way to look at your reading history. Here are a couple of sample visualizations from my data:</p>
<p><img src="/generated/images/2019-resources/instapaper-analytics-tracking-800-dcd8c589c.jpg" srcset="/generated/images/2019-resources/instapaper-analytics-tracking-400-dcd8c589c.jpg 400w, /generated/images/2019-resources/instapaper-analytics-tracking-600-dcd8c589c.jpg 600w, /generated/images/2019-resources/instapaper-analytics-tracking-800-dcd8c589c.jpg 800w, /generated/images/2019-resources/instapaper-analytics-tracking-1000-dcd8c589c.jpg 1000w" /></p>
<p>As you can see a few words popup, Google Data Studio and Goals, two important themes I worked on in late 2018, early 2019. Cool!</p>
<p>Now that we’ve looked at two ways to get our reading history, let’s look at a couple other ways to visualize our data using Google Sheets.</p>
<h3 id="visualizing-your-article-readings-and-highlights-with-google-sheets">Visualizing Your Article Readings and Highlights with Google Sheets</h3>
<p>While not quite as powerful or flexible as Python or a targeting dashboard solution like Google Data Studio or Tableau, Google Sheets remains one of the best place to collect data and can be used to provide good visualizations too, especially once you are automatically collecting data using IFTTT like we are. Let’s look at how to modify the IFTTT Article Read Google Sheet to provide better process the data and starting doing some analytics.</p>
<h5 id="data-processing-your-instapaper-ifttt-in-google-sheets">Data Processing Your Instapaper IFTTT in Google Sheets</h5>
<p>The first thing you are going to want to do is add headers to your Google Sheet. Depending on your setup, it may or may not have these.</p>
<p>Next add another header called “<strong>Timestamp</strong>” below the first row paste in the following code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>=arrayformula( if( len(A2:A), datevalue(regexreplace(A2:A, " at .+$", " ")) + time(regexextract(A2:A, "(\d+):"), regexextract(A2:A, ":(\d+)"), 0), iferror(1/0) ) )
</code></pre></div></div>
<p>Once you’ve added this, it should automatically update and extract the dates. You might need to to update it so it’s in date time format. If so, select that column and go into menu “Format.” Then choose “Number”and then “Date Time.”</p>
<p>What this code is doing is taking the information we got from IFTTT and cleaning it up so it’s a properly formatted field in Google Sheets. It does this by checking if there is any data to work with then doing some regular expression to get just the parts relevant to date and time. We are using an array function so it does this across the entire column and is updated each time a new row is added.</p>
<p>Now that we have a datetime field, it’s a good idea to create different time dimensions we can use to aggregate and visualize. Go ahead and create additional columns:</p>
<ul>
<li>Month</li>
<li>Year</li>
<li>Month-Year</li>
<li>Day of Month</li>
<li>Week Num</li>
</ul>
<p>Then paste some code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>=arrayformula( if( len(A2:A), MONTH(E2:E), iferror(1/0) ) )
</code></pre></div></div>
<p>Replace MONTH with YEAR, DAY and WEEKNUM accordingly.</p>
<p>Because we may end up with multiple years, Month year will be slightly special so add this code instead:</p>
<p>Now that we’ve got data in Google Sheets and created time dimensions like month, year, month-year, etc. Technically, we can create charts and graphs without needing additional dimensions. But it’s better with them since we can use Pivot Tables.</p>
<p>If you don’t already know, Pivot Tables are a way to “flip” the data into aggregated summary tables. In order to create one, go into the menu for “Data” and select “Pivot Table” :</p>
<p><img src="/generated/images/2019-resources/create-pivot-table-in-google-sheets-600-8d30b11f7.jpg" srcset="/generated/images/2019-resources/create-pivot-table-in-google-sheets-400-8d30b11f7.jpg 400w, /generated/images/2019-resources/create-pivot-table-in-google-sheets-600-8d30b11f7.jpg 600w" /></p>
<p>Be sure to update it to include additional rows. Otherwise your reports won’t update as new data is added. After hitting “Create,” you should be directed to a new sheet.</p>
<p>Now to complete your Pivot Table, go to the popup area on the right and add a Row (like Month-Year). Then add a Value (like Timestamp) and be sure to configure “Summarized by” to “COUNTA.” This will count the number of instances.</p>
<p>This should create a chart of articles read by month and it’s easy to add a chart now too:</p>
<p><img src="/generated/images/2019-resources/instapaper-tracking-example-in-google-sheets-800-42ffa694e.jpg" srcset="/generated/images/2019-resources/instapaper-tracking-example-in-google-sheets-400-42ffa694e.jpg 400w, /generated/images/2019-resources/instapaper-tracking-example-in-google-sheets-600-42ffa694e.jpg 600w, /generated/images/2019-resources/instapaper-tracking-example-in-google-sheets-800-42ffa694e.jpg 800w, /generated/images/2019-resources/instapaper-tracking-example-in-google-sheets-1000-42ffa694e.jpg 1000w" /></p>
<p>You may need to use the filter function to exclude or only include certain dates. For example, I used the option to limit empty cells.</p>
<p>Depending on the dimensions you want to look at, you can create a pivot table by week or year. You could also add another dimension by hour of day and look at when you read article too.</p>
<h5 id="creating-a-heatmap-of-daily-articles-read-in-instapaper">Creating a Heatmap of Daily Articles Read in Instapaper</h5>
<p>Obviously Google Sheets is primarily built to be a spreadsheet and it can’t do every kind of data visualization, but it can be tweaked to create a pretty nice looking heat map this this:</p>
<p><img src="/generated/images/2019-resources/instapaper-tracking-heat-map-800-3e9dcc054.jpg" srcset="/generated/images/2019-resources/instapaper-tracking-heat-map-400-3e9dcc054.jpg 400w, /generated/images/2019-resources/instapaper-tracking-heat-map-600-3e9dcc054.jpg 600w, /generated/images/2019-resources/instapaper-tracking-heat-map-800-3e9dcc054.jpg 800w, /generated/images/2019-resources/instapaper-tracking-heat-map-1000-3e9dcc054.jpg 1000w" /></p>
<p>In order to create a heat map, clone or use your original month-year pivot table and add a column for day of the month.</p>
<p>This will create really wide table so first thing you should do select all the columns and then right click and use “Resize Columns.” I used “Fit to Data” but you may need to tweak to get it right the first time.</p>
<p><img src="/generated/images/2019-resources/resize-google-sheets-columns-800-761062bb5.jpg" srcset="/generated/images/2019-resources/resize-google-sheets-columns-400-761062bb5.jpg 400w, /generated/images/2019-resources/resize-google-sheets-columns-600-761062bb5.jpg 600w, /generated/images/2019-resources/resize-google-sheets-columns-800-761062bb5.jpg 800w" /></p>
<p>The next thing we need to do is add some styling. So select the various rows and then add borders and perhaps center the text.</p>
<p>The last step will be to use conditional formatting. Conditional formatting is a way to show gradients of color according to certain rules or formulas. Select all of the cells you want to color and then go to Format and select “Conditional Formatting.”</p>
<p>I used the option “Color Scale” and configured it to use a gradient and set my midpoint with Percentile 30. You may need to adjust it depending on your data. The goal should be to highlight peak reading days and days with low or now reading being white.</p>
<p>That’s it. You now got a heat map showing your article reading for each day of the month. Bravo!</p>
<h3 id="how-to-export-your-article-highlights-into-local-files">How to Export Your Article Highlights into Local Files</h3>
<p>What we want to do is take our stored highlights (from the API or Google Sheets) and make them accessible for our studying and writing. This is the last step for us in augmenting our read-it-later usage experience with Instapaper. This step takes our highlights and moves them into a format we can use for learning and creating.</p>
<p>While highlights are nice by themselves, a good deal of research shows that highlights by themselves aren’t a very good approach to learning. As Dunlosky shows in “Strengthening the student toolbox” (2013), highlights and re-reading just aren’t the most effective ways to learn, and, in fact, they can be problematic, because the instill the illusion of competence. Basically by re-reading and highlighting you get the impression of having learned when the reality is you haven’t.</p>
<p>According to the <a href="http://www.markwk.com/science-of-learning.html">science of learning</a>, there are several better ways to learning. One of my favorites is a form of elaboration and is quite similar to the Feymann Technique. Basically as I read and learn new things, I take notes. These aren’t ordinary scribbled ideas in the margins or your study notes. Instead these so-called <a href="http://www.markwk.com/smart-notes.html">“Smart Notes”</a> are the effortful practice of internalizing a concept and expressing it in your own words in writing. Ideally, you reference where it came from, include an example or two, and link it with other related notes. I find this to be a great practice if knowledge and learning are important to your work.</p>
<p>Now what to do with our highlights? Even though highlights by themselves aren’t enough for learning, they are a great way to get started.</p>
<p>The only problem: How to get these highlights out of Instapaper?</p>
<p>There are a few options. One way would be to setup an IFTTT recipe that pushes highlights into a notebook like Evernote or even Google Docs. Additionally, we can use the API directly to pull highlights or pull them out of Google Sheets.</p>
<p><img src="/generated/images/2019-resources/instapaper-exported-highlights-example-450-4358e6140.jpg" srcset="/generated/images/2019-resources/instapaper-exported-highlights-example-400-4358e6140.jpg 400w, /generated/images/2019-resources/instapaper-exported-highlights-example-450-4358e6140.jpg 450w" /></p>
<p>In <a href="https://github.com/markwk/qs_ledger/blob/master/instapaper/instapaper_data_analysis.ipynb">QS Ledger</a> there is a special script called <a href="https://github.com/markwk/qs_ledger/blob/master/google_sheets/gs_instapaper_highlights.ipynb">gs_instapaper_highlights.ipynb</a> that can serve as a bridge between our spreadsheet of highlights and our local notes. In that script, all you need to do is configure the spreadsheet name and where you want your highlights exported. Then when you run the script it’ll create a separate markdown file for each article.</p>
<p>The exported files each have the basic article info and all of your highlights.</p>
<p>While you can run this using Jupyter Notebook, the easiest and faster way is to run it in the command line with a command like this: <code class="language-plaintext highlighter-rouge">jupyter nbconvert --execute instapaper_highlights.ipynb</code>. Personally, I’ve create a crontab task that runs this hourly as well as an alfred script I can use too.</p>
<p>The end result is that I now have all of my highlights stored locally, and it provides a quick and dirty way to get the necessary info I need on most articles I’ve read.</p>
<p>It’s also a rather seamless experience now to pull up recent articles and their highlights and start <a href="http://www.markwk.com/plain-text-life.html">creating my own learning notes</a>. Highlight might not be the best method for learning, but they are an excellent way to start creating your own elaborative notes, which are.</p>
<h3 id="conclusion-a-dream-for-the-next-futuristic-read-it-later-app">Conclusion: A Dream for the Next Futuristic Read-It-Later App</h3>
<p>Read-it later apps appeared years ago as way to deal with problem of finding something great to read online but not wanting to get distracted and off-topic in the moment. So you save it to read it later. Various tools like Pocket, Evernote and Instapaper solve this problem and many browser can provide similar functionality. They help you save links and provide a better experience for reading by removing the ads.</p>
<p>In this post we looked at my favorite read-it-later tool, Instapaper, which is not only a great basic article reading tool but has a few special features that make it even better. Namely, Instapaper provides the capacity to track what you read and use that data to see trends over time. This is easily accomplished using its integration with IFTTT or with its API and some coding. Additionally, Instapaper allows you to highlight parts of the articles you read. You can then export those highlights too. This combination of features (reading, tracking and highlighting) makes it an excellent tool of choice for anyone looking to augment their digital memory and article reading, especially towards learning and creation.</p>
<p>To put it another way, unlike many apps, Instapaper provides an example where technology can be used to support better focus and improved learning. By applying some spreadsheet or data science skills, we can become more aware of how and what we read. We can also create some interesting data storytelling about our reading behaviors too.</p>
<p>For example, I’ve used Pocket, Kindle, GoodReads and Instapaper over the years to essentially track nearly everything I read and use. These tracking stats and visualizations have changed how I quantify my habits and empowered a tighter learning cycle between what I read and the smart notes I create.</p>
<p>Overall, I’m quite happy with my current article reading and tracking setup, but being a reader of sci-fi and future thinking scientists, I can’t help but wonder: What else might these reading apps become one day?</p>
<p>Right now these read-it-later apps are essentially productivity tools. They largely improve the experience of reading online articles, and they help you remember something to read <em>later</em> and look up what you read <em>before</em>. They are by no means “smart” or “intelligent” and they aren’t particularly personalized either.</p>
<p>If we want to improve the future of our reading and learning apps, the first step is to infuse them with a degree of personalization. In his book, “The Inevitable,” Kevin Kelly writes about the role of technology in human lives and society in the future, including tracking, screening, flowing, etc. One stands out when it comes to a future reading app and that’s filtering.</p>
<p>According to Kelly, we filter by gatekeepers, by intermediates, by curators, by brands, by government, by our cultural environment, by our friends, and by ourselves. One big on is Google. “Google is the foremost filterer in the world, making all kinds of sophisticated judgments about what search results you see. In addition to filtering the web, it processes 35 billion emails a day, filtering out spam very effectively, assigning labels and priorities. Google is the world’s largest collaborative filter, with thousands of interdependent dynamic sieves. If you opt in, it personalizes search results for you and will customize them for your exact location at the time you ask.”</p>
<p>What Kelly imagines is an alternative form of personalized filtering. Using a combination of artificial intelligence, our history, and even specified parameters, he foresees that even greater personalization and tailoring of this filtering. He postulates that there will services and bots you use to make this happen. The AI going out a finding relevant information and even processing it too. For example, I’d love for my reading app to look at my reading trends and recommend related content on these subjects. It might see that I’m struggling to learn something and advice me to check out one of the best guides about a certain aspect.</p>
<p>This kind of advanced, AI-powered filtering might take time to arrive. If you imagine the computational resources spends on its search function and then think about scaling that to a single human, you start to realize the size of the problem.</p>
<p>Fortunately some simple that could be added would be clustering of articles and topics. After reading thousands of articles and books over the years, it’s unfortunate that I can’t go beyond word clouds to visualize these trends. What I would like to see is a smart clustering showing past, present and future articles and content on areas I’m reading and learning.</p>
<p>One other area I’d like to see my reading apps improve upon is acting as a closer guide to my learning. I’ve been thinking about this on my <a href="www.podcasttracker.com">podcast tracking service</a> as a gap between listening to a great podcast and failing to use it effectively for learning. One step might be quizzes and follow up questions. But it seems obvious that when we use these reading tools, it’s likely because we value that information towards a creative or learning goal. So the question becomes how can these podcast and article tracker help us to learn better?</p>
<p>The last component for a better reading app would be “intelligence.” The current tools are for productivity, but the future ones might take inspiration from the sci-fi writer Neal Stephenson’s book, <em>The Diamond Age</em> (2003). The book is a wonderful combination of story and futuristic thinking with one of its chief characters being a special kind of book called “A Young Ladys Illustrated Primer,” which was intended to educate a girl to think for herself. The book has artificial intelligence and a real agent that guides the girl through a number of immersive, learning challenges. Originally developed for an upper class baron, the inventor decides to smuggle out a copy and share it with someone in the lower class. Ultimately the hope is to bring more access to education to everyone, but strikingly it’s also an amazing example of how to think about our own learning and reading apps.</p>
<p>While the next evolution for read-it-later apps might be targeted recommendations and better visualization of our reading history, personally my future dream is for much more: a personalized filter of great content, an intelligent reading buddy that can nudge me to read the right things at the right time, and hopefully one day an interactive guide to tailored content and challenges so I too can think for myself.</p>
<p><br /></p>
<p><br /></p>
<h5 id="references">References:</h5>
<ul>
<li>Dunlosky, J. (2013). Strengthening the student toolbox: Study strategies to boost Learning. <em>American Educator</em>.</li>
<li>Kelly, K. (2016). The Inevitable.</li>
<li>Stephenson, N. (2003). The Diamond Age. Spectra.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Are you facing a never ending pile of articles to read? Have you ever remembered this amazing post you recently read but forgotten the title or URL? Or are you just hoping to augment and improve how you read and learn stuff online using technology?How I Write: My Favorite Tools and Apps for Writing2019-08-05T00:00:00-07:002019-08-05T00:00:00-07:00http://www.markwk.com/writing-tools-2<p>So, you want to write? And you’re looking for a few tools to make your writing easier, better organized or adapted for a new publishing format?</p>
<p>I’ve written and published over 300 blogs, articles and book chapters over the last several years. While the tools aren’t as important to me as the time, attention and process I put into writing, I’ve come to find good tools can empower and smooth out several aspects of my research and creative process.</p>
<p>These are the tools that put me into writing zen!</p>
<p>Whether you are working on the next great American novel, creating a work report, penning a poem, or just striving to get a shareable blog post done, here are my favorite tools for writing for writers.</p>
<p><br /></p>
<p><em>NOTE: This list is Version 2.0 (circa August 2019). If you are interested in my writing tools from early 2018 (which included Ulyssess and Evernote), see <a href="http://www.markwk.com/writing-tools.html">Version 1.0 of this post</a></em>.</p>
<!--more-->
<p><br /></p>
<h3 id="my-core-writing-tools">My Core Writing Tools</h3>
<p>These are the true workhorse tools for my writing and creative process: The Archive, Typora, and Bookends.</p>
<p>These tools are also at the heart what I call <a href="http://www.markwk.com/plain-text-life.html">The Plain Text Life</a>, a setup of plaintext files I use for note-taking, writing and life organization and <a href="http://www.markwk.com/track-writings.html">tracking my notes</a> it too.</p>
<h4 id="the-archive-a-smart-notes-based-notes-organizer">The Archive: A “Smart Notes”-based Notes Organizer</h4>
<p><img src="/generated/images/2019-resources/the-archive-writing-tool-800-b1c5d8303.jpg" srcset="/generated/images/2019-resources/the-archive-writing-tool-400-b1c5d8303.jpg 400w, /generated/images/2019-resources/the-archive-writing-tool-600-b1c5d8303.jpg 600w, /generated/images/2019-resources/the-archive-writing-tool-800-b1c5d8303.jpg 800w" /></p>
<p><strong>Solves: Tag-based system for organizing and connecting together all of the notes I use in learning, writing and other creative projects</strong></p>
<p>The Archive has been my go-to tool for managing and organizing all of my notes and creative drafts for awhile. Inspired by the book, <a href="http://www.markwk.com/smart-notes.html">How to Take Smart Notes</a>, I decided to <a href="http://www.markwk.com/migrate-evernote-plaintext.html">migrate off of Evernote</a> and started using The Archive along with Typora for nearly everything I write or take notes on.</p>
<p>Unlike many other note-taking and writing apps, The Archive acts more like a file and tags organizer on-top of a simple text editor. All of the files are separate, plaintext markdown file.</p>
<p>By default, The Archive has a nice look and feel. Very writer-ly. It also has adjustable themes and fonts if it’s not to your taste.</p>
<p>To give you a quick orientation, on the right is the main editor panel for writing. In-between, on the left is a list of files. This could be a list of all files or a sub-list of files according to a tag or cross-linked. Subtly at the very top is a search bar where you can look up file names, tags, etc. On the far left with icons are customizable buttons that you can use to segment your files. For example, I have a button for my morning journal tag, one for my summary notes, and two others for working and general drafts.</p>
<p>You type notes in markdown and, according to the syntax, it provides some general highlighting. Additionally, anytime you put a hashtag gets converted into a link. When you click those tag links, it does a search on files with that tag and displays them in the left files panel.</p>
<p>The real power of The Archive is thinking about it like a Wiki. In fact anytime you create links between things it becomes a wiki. So, like in a Wiki, when you create links between notes using double brackets, The Archive makes it simple to crosslink between different notes. This ability to link between multiple notes make it a powerful framework for an interconnected knowledge network.</p>
<p>Here is an example of a summary note on creativity, which links to multiple individual notes I have created on this topic:</p>
<p><img src="/generated/images/2019-resources/the-archive-screenshot-800-8f63ce9b9.jpg" srcset="/generated/images/2019-resources/the-archive-screenshot-400-8f63ce9b9.jpg 400w, /generated/images/2019-resources/the-archive-screenshot-600-8f63ce9b9.jpg 600w, /generated/images/2019-resources/the-archive-screenshot-800-8f63ce9b9.jpg 800w, /generated/images/2019-resources/the-archive-screenshot-1000-8f63ce9b9.jpg 1000w" /></p>
<p>A discussion on how to organize your notes and writings is a bit outside the scope of this post, but here is a quick example of how I use tags in my creative writing workflow. Basically, my primary tags are: draft-idea, draft-working, draft-working-focus, draft-publishable, and Blog-Published. I use these five tags on on drafts, which cover my workflow process from idea stage to draft stage to publishing. Depending on how you want to draft, write and publish, you can setup an appropriate tagging convention for your needs. Similarly there is no real limits or forced conventions on tagging overall in The Archive.</p>
<p>For me the main features I enjoy about The Archive are its clear interface for typing and editing markdown and its smooth interaction between notes using wiki-style links and tags. The segmented lists make it easy to quickly see what notes you have on a topic or tag.</p>
<p>The main limitations I find with it are lack of inline images and inability to have multiple windows open at the same time. Fortunately, The Archive provides a shortcut and option called “Open with External Editor,” which allows you open files a separate editor and largely solves both of these limitations for me.</p>
<p>The Archive is available for Mac with a long free trial period before a one-time fee, which covers many features to come.</p>
<h4 id="typora-best-looking-markdown-editor-on-mac">Typora: Best Looking Markdown Editor on Mac</h4>
<p><img src="/generated/images/2018-resources/evernote-to-plaintext-exported-files-example-800-f43d78d03.jpg" srcset="/generated/images/2018-resources/evernote-to-plaintext-exported-files-example-400-f43d78d03.jpg 400w, /generated/images/2018-resources/evernote-to-plaintext-exported-files-example-600-f43d78d03.jpg 600w, /generated/images/2018-resources/evernote-to-plaintext-exported-files-example-800-f43d78d03.jpg 800w, /generated/images/2018-resources/evernote-to-plaintext-exported-files-example-900-f43d78d03.jpg 900w" /></p>
<p><strong>Solves: Minimalist Markdown Text Editor with a Clean Interface, Embedded Images and Multiple Export Options</strong></p>
<p>There are awful lot of markdown editors out there today on any and all platforms. Typora in my opinion is one of the best looking and has a very smooth user experience.</p>
<p>Typora came to my attention for two features. First, unlike other editors, it bridges the line between raw plaintext and a WYSIWYG (“What You See Is What You Get”) editor. So, you can see images inline and styles show up as you write, instead of in a separate window or when you export. Underneath though it’s all just markdown text.</p>
<p>Second, Typora stands out in terms of its layout and typography. It’s not quite as coder-oriented as many other markdown tools. Basically, it looks like any other modern Mac application with nice fonts, margins and spacing.</p>
<p>Since it isn’t pure text, you’ll need to learn a few shortcuts to make the experience smooth, but once you get the hang of it, it really is a great way to write and edit plain text documents.</p>
<p>One of my favorite features with Typora is its ability to embed images. This allows me to include local and external images that are displayed seamlessly, much like other full-featured note-taking tools.</p>
<p>Typora is available for Mac, Windows and Linux and is under active development.</p>
<h4 id="bookends">Bookends:</h4>
<p><img src="/generated/images/2018-resources/bookends-reference-manager-for-writers-800-9b760666c.jpg" srcset="/generated/images/2018-resources/bookends-reference-manager-for-writers-400-9b760666c.jpg 400w, /generated/images/2018-resources/bookends-reference-manager-for-writers-600-9b760666c.jpg 600w, /generated/images/2018-resources/bookends-reference-manager-for-writers-800-9b760666c.jpg 800w" /></p>
<p><strong>Solves: Find and Store Research References, Generate a Formatted Bibliography</strong></p>
<p>Bookends is an example of reference management software. Its primary users are academics and researchers, who need to collect and organize their research and to create properly formatted bibliographic information for their publications. There are a number of options in this software category like Papers and Zotero, and plenty of fans of each. After surveying a few tools, I elected to use Bookends as my reference and bibliography manager, and, in spite of a few initial hiccups, I’ve been really happy with it.</p>
<p>The primary purpose of Bookends is to help you keep track of reference and citation information on academic papers and books as well as internet sources. Since I read a good amount of formal research articles and books, I dump that info into Bookends. Additionally, when there is a PDF, I attach it to its reference too.</p>
<p>It’s a bit tedious getting started (especially if it’s been over a decade since you left graduate school), and you’ve forgotten how to create a work cited page. Fortunately, Bookends provides default fields so you know what to collect on each article or book. It also has a few nice automation tools. For example, you can import complete book references from Amazon and article and journal citations from Google Scholar, pubMed, or several other academic databases. This can really speed up setting up your initial database of citations.</p>
<p><img src="/generated/images/2018-resources/bookends-reference-manager-for-writers-search-750-8e9885b62.jpg" srcset="/generated/images/2018-resources/bookends-reference-manager-for-writers-search-400-8e9885b62.jpg 400w, /generated/images/2018-resources/bookends-reference-manager-for-writers-search-600-8e9885b62.jpg 600w, /generated/images/2018-resources/bookends-reference-manager-for-writers-search-750-8e9885b62.jpg 750w" /></p>
<p>Often what I do is use Google Scholar for the search part and then save and export my citations from there. It’s easy to then bulk import. them into Bookends.</p>
<p>Once you have added your references, Bookends makes it easy to then export a properly formatted bibliography or works cited. Simply select the articles you are using and then export it. Bookends provides nearly a dozen standard formats by default, and it’s easy to create or adapt your own.</p>
<p>Additionally, Bookends integrates with any number of text editors and tools, so you can create smart references. This makes adding references inside the text you are writing easy. Basically you copy the reference out of Bookends and then paste it as special tag into your text editor. Finally, when you need to render the completed manuscript, the text you wrote is combined with the references you’ve added from Bookends to generate the final text. Personally, I prefer using the full formatted citation, rather than a smart reference.</p>
<p>Since using Bookends for nearly a year, my references have grown from a few dozen to over 2000. This is not something I could manage in my head or without a tool like this. Having a reference manager tool has been extremely helpful in my learnings and for science-based writings online and in book form.</p>
<p>Bookends might not be a tool for all types of writing, but if you do academic and research style writing, a reference manager is a must-have.</p>
<h4 id="my-design-tools-for-writing">My Design Tools for Writing</h4>
<p>I also dabble in a bit of design work. Design considerations go a bit beyond this post, but since I typically create graphics for everything I publish, it’s worth mentioning the tools I use to make it happen.</p>
<p>I’ve used lots of different design tools over the years, but ironically one of my favorite tools to create quick graphics and blog covers is with a presentation app, like Keynote for Mac. These tools are specifically designed for creating slideshows and presentations, but they also work great for throwing together a few images, graphical elements and text too.</p>
<p>Additionally, while they are a bit “old school,” I also use Adobe Photoshop and Illustrator to help me in creating graphics and editing photos for my writings.</p>
<p>Looking back, I’m definitely glad I invested a couple months of study time to become proficient in basic design in general and with a tool like Photoshop. These basic designs skill mean that I can create decent looking images for my needs, and I‘ve saved a lot of time and money too.</p>
<h3 id="supplemental-tracking-and-productive-tools-for-writers">Supplemental Tracking and Productive Tools for Writers</h3>
<p>While not critical for how and what I write, here are a few more tools that help me to track my writing, to stay productive and organized while writing, and to improve and automate my writing workflow.</p>
<h4 id="tracking-my-word-count-with-word-counter-for-mac">Tracking My Word Count with <em>Word Counter for Mac</em></h4>
<p><img src="/generated/images/2019-resources/wordcounter-tracking-stats-800-73db0b4c8.jpg" srcset="/generated/images/2019-resources/wordcounter-tracking-stats-400-73db0b4c8.jpg 400w, /generated/images/2019-resources/wordcounter-tracking-stats-600-73db0b4c8.jpg 600w, /generated/images/2019-resources/wordcounter-tracking-stats-800-73db0b4c8.jpg 800w" /></p>
<p>I’m a bit of an obsessive <a href="http://www.markwk.com/category/track-everything/">tracker</a>, and I use a lot of tracking tools. I use two different ways to <a href="http://www.markwk.com/track-writings.html">track my writing</a> itself. For my typed words per day, I use Word Counter. It’s a tool that keeps track of my daily word count in different applications.</p>
<p>It might be a vanity metric, but I enjoy seeing my previous day’s word count and looking up today’s word count too. As a writer, it’s good to know that I managed to write some words each and every day, and as a dabbler in data science and data visualization, it’s a cool data point to explore.</p>
<p>I even created an Alfred Workflow for quick display of your daily writing stats. <a href="https://github.com/markwk/alfred-workflow-wordcounterapp">Download it here</a>.</p>
<p>In case you were wondering, as one friend reviewer asked, which tool do I use the most? In recent months, the time spent in Typora and The Archive is pretty even with a slight edge to Typora, according to my time tracking.</p>
<h4 id="tracking-my-notes-and-drafts-with-git">Tracking My Notes and Drafts with Git</h4>
<p>The other way I <a href="http://www.markwk.com/track-writings.html">track my writings</a> is I monitor the actual files I use. Since all of the files I use to take notes and write are plaintext files, I decided to leverage Git, a software management tool and tracker, to track all of my writing files. It’s really a pretty simple setup that has a daily script that looks at the changes in all of my writing and project note files, stores it into a repo and records some high-level statistics, like words per day, lines and files changed, and total number of files among others.</p>
<p><em>NOTE: You can find the bash script code https://github.com/markwk/writing-tracker or read my full write-up <a href="http://www.markwk.com/track-writings.html">here</a>.</em></p>
<h4 id="alfred-mac-automation-tool">Alfred: Mac Automation Tool</h4>
<p><img src="/generated/images/2018-resources/alfred-word-search-500-8328a9e84.jpg" srcset="/generated/images/2018-resources/alfred-word-search-400-8328a9e84.jpg 400w, /generated/images/2018-resources/alfred-word-search-500-8328a9e84.jpg 500w" /></p>
<p>Alfred is a powerful Mac automation tool. If you work on a Mac, you should be using Alfred. Alfred provides a faster way to do various things, like simplifying Web Searches and opening specific documents or applications. You can also customize it to search directly to certain sites, like Google Scholar or your local library. There are hundreds of add-on workflows that you integrate different tools and services with Alfred. I describe Alfred as way of bringing command line productivity to the forefront of how you work on Mac.</p>
<p>For writing purposes, I use Alfred to make synonyms lookups seamless, custom copy and paste commands, query stock images for designs, and to query my daily word counts from Word Counter without opening up another app. You can use my Alfred Workflow for Wordcounter <a href="https://github.com/markwk/alfred-workflow-wordcounterapp">here</a>.</p>
<p>When I’m struggling get going or need a way to simple do a session, I also use Alfred as a simple pomodoro timer. For example, if I want to write for 25 or 35 minutes, before taking a break or walk, it’s a few keystrokes to start a timer like this.</p>
<p><img src="/generated/images/2018-resources/alfred-timer-632-528a432c2.jpg" srcset="/generated/images/2018-resources/alfred-timer-400-528a432c2.jpg 400w, /generated/images/2018-resources/alfred-timer-600-528a432c2.jpg 600w, /generated/images/2018-resources/alfred-timer-632-528a432c2.jpg 632w" /></p>
<h4 id="my-key-productivity-tools-streaks-todoist-toggl-and-google-calendar">My Key Productivity Tools: Streaks, Todoist, Toggl, and Google Calendar:</h4>
<p>I believe most work gets done, because we simply show up and do the work. <strong>Writing is not really an act of inspiration or waiting for the muses; it’s an act of sitting down and doing it.</strong> The best writers have built up the habit of “showing up and writing,” and they do it consistently.</p>
<p>I strive for the same mindset as a writer. I feel like I continue to grow as a writer through constant repetition and opportunities to express myself.</p>
<p>I use a number of techniques and processes to make sure I stick to my writing habits. When I am a bit off on a routine, I use Streaks app to <a href="http://www.markwk.com/2016/10/habit-tracking-for-quantified-self.html">track my habits</a>. Blocking out time in my calendar to write helps too, and I use Google Calendar to schedule my regular writing times, like I would do any other meeting (except it’s a meeting with just myself). I use <a href="http://www.markwk.com/gtd-with-todoist.html">Todoist</a> to collect all the various writing tasks I need to do to complete a project and it helps me <a href="http://www.markwk.com/task-tracking-with-todoist.html">track my tasks</a> too. Finally, I use Toggl to <a href="http://www.markwk.com/time-tracking-tools.html">track my time</a>, including my writing sessions and work as a whole.</p>
<p><img src="/generated/images/2019-resources/writing-time-2017-vs-2018-800-7929b34f8.png" srcset="/generated/images/2019-resources/writing-time-2017-vs-2018-400-7929b34f8.png 400w, /generated/images/2019-resources/writing-time-2017-vs-2018-600-7929b34f8.png 600w, /generated/images/2019-resources/writing-time-2017-vs-2018-800-7929b34f8.png 800w, /generated/images/2019-resources/writing-time-2017-vs-2018-819-7929b34f8.png 819w" /></p>
<p>All combined these tools ensure I’ve committed habitual time to writing and help me keep track of how and how much I write. The most important thing though is that I commit to writing and carve out time to do it. “Magically,” when I do, words get written, blog posts get published, and chapters in my books advance.</p>
<h3 id="conclusion-find-your-tools-but-commit-to-a-schedule-and-focus-on-the-process">Conclusion: Find Your Tools, But Commit to a Schedule and Focus on the Process</h3>
<blockquote>
<p>“Writing is thinking. To write well is to think clearly. That’s why it’s so hard.” - David McCullough</p>
</blockquote>
<p>I’m passionate about writing. While I’ve managed to complete a lot of writing over the years, I’m still growing as a writer. There are days when the writing goes well, and others where I struggle. Fortunately, I’ve committed to writing as a lifetime activity, and I look at each day as opportunity to show up and put in the time in my craft as a writer.</p>
<p>In this post, we looked at several of my favorite tools for writing. Specifically my favorite two are The Archive and Typora, which are great especially if you are into markdown format. I’ve done a lot of great writing in and with just those two tools.</p>
<p>Beyond those, I use a number of organizational tools that help me stay productive and organized as a writer. For example, I really like my current reference manager, Bookends, and my word count tracker, Word Counter for Mac. They aren’t indispensable, but they sure do improve how I like to write. Additionally I’m a huge fan of Alfred, and a few workflows that make web searches and thesaurus look-ups more seamless as well as pasting file paths as markdown images.</p>
<p>To help me manage my time and writing, I use a number of productivity tools. I believe it’s critical to leverage what you like to manage your time, tasks and calendar. For me, I use and like Google Calendar to schedule weekly blocks of time for writing, I create tasks in Todoist for writing, planning and design work, and I use Toggl to keep track of my weekly writing time. These all are provide consistent reinforce to my habit of writing. While it’s low-tech, each week I try to set one writing goal and score it along with my other <a href="http://www.markwk.com/goal-scoreing.html">weekly goals</a>.</p>
<p>In the end, I encourage you to find your tools as a writer, but don’t procrastinate too much on tools at the expense of avoiding the act of writing. Don’t fall into a trap of thinking there is a better tool or process out-there and wasting away your writing time searching and getting setup. Ultimately what matters most is committing to writing. The best writers have managed to create even before we had computers. Their secret: showing up and writing. So, focusing on the process and keep writing.</p>
<p><br /></p>
<p>Good luck and happy writing!</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}So, you want to write? And you’re looking for a few tools to make your writing easier, better organized or adapted for a new publishing format?No YouTube: 30-Day Challenge2019-08-02T00:00:00-07:002019-08-02T00:00:00-07:00http://www.markwk.com/a-month-without-youtube<p>Over the last 6 months or so, I’ve been <a href="http://www.markwk.com/youtube-tracking.html">tracking my Youtube usage</a>. I came to discover that I was spending on average about an hour a day watching YouTube videos. While there are a lot of things I love about YouTube, it comes with some negatives. So, for one month, I gave up YouTube.</p>
<p>Admittedly, at times I missed watching videos online, especially comedy and some news updates, but <strong>largely I found life better without YouTube than with it</strong>. It showed me just how complicated one’s relationship can be with technology and various digital services. YouTube and many other services are designed to hook us and keep us using them. They are designed to be addictive. Sadly, as of now, we can’t adjust these technologies to fit our ideal way of using them either.</p>
<p>30-day challenges are a great way to experiment with something in your life. Month-long challenge is largely about discovery and are burdensome than full-on goals or building new habits. By trying something out for a month, you can figure out if it’s something you want to continue going forward.</p>
<p>Most of the routines I now have started as short trials or challenges, like running, daily morning pages, and time tracking. I’ve done a number of month-long experiments over the years, and I am especially a fan of tracking them too. In fact, some of my 30-day challenges were tracking-specific challenges, like what I did with <a href="http://www.markwk.com/mood-tracking-experiment.html">mood tracking</a> and <a href="http://www.markwk.com/poop-tracker.html">tracking my bowels</a>.</p>
<p>I had a couple motivations for this 30-day No YouTube challenge. The first was to spend less time binge watching random online videos and more time reading, writing, and other personal projects. I figured I’d lose out on some amount of news and content, but that seemed okay to me. I hypothesized that an extra 8 hours a week would mean more books and articles read and possibly a bit more time on the computer. Basically, the main thing I expected to see a big difference in my time logs and media usage. This wasn’t exactly what I find.</p>
<p>Like any technology, YouTube has its pro’s and con’s. Experiments like these provide a good way to figure out what works for you.</p>
<p>In this post, I want to share about about my month-long No YouTube challenge. To give a bit of context, in the first part, I’ll talk about what is a 30-day challenge and some of the benefits of this kind of short-term discovery type of goal. After that, I’ll briefly share how and why I did one on YouTube Watching. We will then dig into the my data and tracking logs to figure out what what changed (and what didn’t!) over the last month. Finally, I’ll conclude with some general observations on the challenge, technology, and if and how I’ll use YouTube going forward.</p>
<p><br /></p>
<p><strong>NOTE: If you are interested in tracking your YouTube usage, I’ve written a <a href="http://www.markwk.com/youtube-tracking.html">complete guide to YouTube Tracking</a>.</strong></p>
<!--more-->
<h3 id="what-is-a-30-day-challenge-and-why-do-them">What is a 30-day challenge? And Why Do Them?</h3>
<p>Goals and goal-setting are some the most effective cognitive strategies we can use for behavior change. As I’ve talked about in my on-going series on the <a href="http://www.markwk.com/category/science-of-goals/">science of goals</a>, goals exist as a <a href="http://www.markwk.com/multistage-goal-pursuits.html">multistage pursuit</a> wherein among other elements you have the goal intention or goal-setting, which refers to what you want to achieve, and the goal pursuit or goal striving, which are the actions you take in pursuit of that goal.</p>
<p>Much of research on goals focused on one of these two ends, either goal-setting or the goal pursuit and different strategies you can use. <a href="http://www.markwk.com/science-of-goals.html">Goal Setting Theory (GST)</a> has shown that <em>how</em> you set goals modifies your performance and demonstrated that harder, more specific goals are more effective than vague or easy goals. When comes to the goal pursuit itself, two effective techniques are mental contrasting and implementation intentions (if-then plans).</p>
<p>A 30-day challenge is a slightly different kind of goal. The main point is to try something new for awhile and figure out if it works for you or not. The intention should be one of discovery and learning, rather than expecting to develop a new habit in such a short amount of time. In fact, a good amount of research indicates that most habits take more than 30 days to develop.</p>
<p>Also unlike the <a href="http://www.markwk.com/science-of-goals.html">science of goal-setting</a> that focuses on longer-term objectives and harder challenges, 30-day challenges are generally less ambitious and come with less of an obligation too. They are, by design, less of an automatic thing you do and more of a scheduled or willed action we are experimenting with. A few good examples of 30-day challenges are reading daily, giving up caffeine, exercising for 30 minutes, or writing in a journal. Basically, you try it for a while and see if you like it.</p>
<p>All things considered, 30-day challenges can can still can be an excellent way to figure out of if a certain lifestyle change, technology, or something else is right for you. Or, like my example, if life <em>without</em> a certain technology is right.</p>
<p>One of the best things about 30-day challenges is they take place almost entirely within the “honeymoon” period, meaning it feels new and fresh, so your curiosity and openness get peaked. Their shorter nature makes you more conscious and observant too. They encourage us to track (especially fitness or reading challenges) and often times you’ll find yourself aware of the changes.</p>
<p>The natural way a month challenge fits with our calendars is a good thing too. There is a a nice rhythm to them. You get sparked by a new month to try it, and as the month comes to an end, you organically become reflective.</p>
<p>Whether or not you are a hardcore <a href="http://www.markwk.com/category/track-everything/">tracker</a> or into <a href="http://www.markwk.com/quantified-self-mind-map.html">quantified self</a> like me, once completed 30-day challenges are generally pretty easy to do a comparison. All you have to do is look at the previous month or two, and ask yourself what changes you felt over that period. If you have tracking logs a simple before and after comparison should be enough.</p>
<p>My advice when it comes to month-long challenges is to first a list of all the things you want to try at some point. Then pick one and do it for a month. Don’t make them too hard at the beginning. If you want to take advantage of the science of goals, then make a bit of extra effort to define your goal better so that it is tied into a trigger. So, for example, if you want to exercise daily, you should set your goal something like this:</p>
<blockquote>
<p>After getting home from work, I’ll go for a run or workout at the gym. If it’s after 9pm and I realize I haven’t exercised yet today, then go for a short 15 or 20 minute walk around the neighborhood or a quick session of stretching or yoga.</p>
</blockquote>
<p>This sort goal setting is called <strong>implementation intentions</strong> or more informally if-then plans. There is a lot of strong research showing the effective of these cognitive strategies at following through on our goals and habits (Gollwitzer & Sheeran, 2006; Gollwitzer, 1999). Essentially what you are doing is mentally linking an activity with a situation or trigger. So when that situation arises, you will simply do that act rather than thinking about it or deploy willpower to force yourself.</p>
<p>There are other cognitive strategies you can use, like blocking out the time in your calendar, finding a buddy to work on the challenge together, or using a <a href="http://www.markwk.com/goal-tracking-with-airtable.html">goal tracker</a>. Since the action is likely new, I find a <a href="http://www.markwk.com/2016/10/habit-tracking-for-quantified-self.html">habit tracker</a> to be the best way to ensure I stick to the new routine.</p>
<p>Now that we’ve looked at what is a 30-day challenge and some of its benefits, let’s take a look at my recent experiment.</p>
<h3 id="my-30-day-no-youtube-challenge">My 30-Day No YouTube Challenge</h3>
<blockquote>
<p>“Nobody exists on purpose, nobody belongs anywhere, everybody’s going to die. Come watch tv.”</p>
<p>Rick from the TV show, <em>Rick and Morty</em>.</p>
</blockquote>
<p>I’ve been a regular user of YouTube for a long time. The platform gives me targeted content for free anytime anywhere I want it, which is great. There are tons of educational videos, and content that makes me laugh, smile and think. I’ve used the platform to learn about statistics, economics and data science. It’s also where I can to ponder the <a href="https://www.youtube.com/watch?v=hWFDHynfl1E">Philosophy of Rick and Morty</a>.</p>
<p>But let’s be honest. YouTube is a channel that delivers entertainment sandwiched between advertising. It’s objective is to maximize the amount of time you spend on it so they can maximize your exposure to ads. By most accounts, YouTube is one of the most successful online platforms in the so-called attention economy.</p>
<p>As I share in my <a href="http://www.markwk.com/youtube-tracking.html">Guide to YouTube Tracking</a>, there are a few ways to track your usage. Personally, I track it once a week by opening up the YouTube app where there is a time report and manually logging my weekly YouTube time into my <a href="http://www.markwk.com/data-driven-weekly-review.html">data-driven weekly review</a> spreadsheet. Here is a breakdown of my recent usage:</p>
<p><img src="/generated/images/2019-resources/youtube-watching-time-log-800-686267fac.png" srcset="/generated/images/2019-resources/youtube-watching-time-log-400-686267fac.png 400w, /generated/images/2019-resources/youtube-watching-time-log-600-686267fac.png 600w, /generated/images/2019-resources/youtube-watching-time-log-800-686267fac.png 800w, /generated/images/2019-resources/youtube-watching-time-log-862-686267fac.png 862w" /></p>
<p>The big negatives for me with YouTube are the time suck, addictive tendency, binge usage, high arousal, and sleep loss. I also find it occasionally acts as an emotional retreat, when I’m feeling lonely or a little depressed. Basically, periodically I binge a bunch of videos in a long session, and this tends to happen when I’m a bit off. Unfortunately, this tends to result in not going to bed on time and less sleep only makes it worse.</p>
<p>Moreover when I travel, I sometimes fall into some bad habits. One of those habits is YouTube binging. It happens when I’m not traveling too. I don’t have a specific count of how often, but my guess it happens roughly every week or two where I end up not going to bed at a good time and spending 3 to 4 hours straight watching endless series of videos on YouTube. While it might start off as educational, it often ends up being a lot of time for not much gained.</p>
<p>Coming back home after roughly a month of travel, I decided it might be interesting to take a break. Inspired by another tracking buddies monthly challenges, I did my own: 30 Days without YouTube.</p>
<p>Going into my YouTube Free Month, I had already spent roughly 213 hours on YouTube this year. <strong>I was averaging 8 hours and 53 minutes a week, which equated to 1 hour and 16 minutes per day</strong>. Like it or not, that’s a lot of time.</p>
<p>Setting up this challenge wasn’t hard I simply deleted the YouTube apps on all my devices. Since I don’t use YouTube on the computer, this was enough to stop me not from using it.</p>
<p>Interestingly, initially I had these habitual moments in the evenings and at breakfast where I felt the urge to watch a video. Like or not, I’d created triggers in my day where I watched YouTube. Fortunately, after a few days my habits changed, and I didn’t really notice it. I turned on some music at breakfast, watched the news on TV or did a bit of light reading. Being a constant learner, I generally don’t struggle to find ways to occupy my mind. I just needed to a new medium.</p>
<p>So, it’s been a month now. What changed and what didn’t? What does the data say?</p>
<h3 id="data-analysis-things-that-didnt-change">Data Analysis: Things That Didn’t Change</h3>
<p>Interestingly my data didn’t quite look like I expect it to.</p>
<p>Here are a couple of charts that attempt to put this month in perspective, based on my device usage and a few other parameters I track:</p>
<h5 id="weekly-time-on-devices-no-major-changes-in-phone-and-tablet-usage">Weekly Time on Devices: No Major Changes in Phone and Tablet Usage</h5>
<p><img src="/generated/images/2019-resources/device-usage-time-logs-800-2377b9c12.png" srcset="/generated/images/2019-resources/device-usage-time-logs-400-2377b9c12.png 400w, /generated/images/2019-resources/device-usage-time-logs-600-2377b9c12.png 600w, /generated/images/2019-resources/device-usage-time-logs-800-2377b9c12.png 800w, /generated/images/2019-resources/device-usage-time-logs-843-2377b9c12.png 843w" /></p>
<p><em>NOTE: Additional device data I added to my my <a href="http://www.markwk.com/data-driven-weekly-review.html">data-driven weekly review</a> include ScreenTime from iPhone and iPad. My computer usage comes from RescueTime.</em></p>
<p>This device usage chart shows variation due to up and down work and travel. I work in various creative and professional sprints too.</p>
<h5 id="heatmap-of-articles-read-in-recent-months-according-to-instapaper">Heatmap of Articles Read in Recent Months, According to Instapaper</h5>
<p><img src="/generated/images/2019-resources/article-reading-heatmap-800-58b2625bc.png" srcset="/generated/images/2019-resources/article-reading-heatmap-400-58b2625bc.png 400w, /generated/images/2019-resources/article-reading-heatmap-600-58b2625bc.png 600w, /generated/images/2019-resources/article-reading-heatmap-800-58b2625bc.png 800w, /generated/images/2019-resources/article-reading-heatmap-1000-58b2625bc.png 1000w" /></p>
<p><em>NOTE: I generated this using the Instapaper and its integration with IFTTT. Data was stored into a Google Sheet and I used a pivot table with some conditional color formatting to make it look like a heatmap.</em></p>
<p>This articles read chart shows an increase in article read during this challenge, but it wasn’t markedly different from the overall trajectory.</p>
<h5 id="kindle-highlights-per-month-less-highlights">Kindle Highlights Per Month: Less Highlights</h5>
<p><img src="/generated/images/2019-resources/monthly-kindle-hightlights-727-8a980dc7c.png" srcset="/generated/images/2019-resources/monthly-kindle-hightlights-400-8a980dc7c.png 400w, /generated/images/2019-resources/monthly-kindle-hightlights-600-8a980dc7c.png 600w, /generated/images/2019-resources/monthly-kindle-hightlights-727-8a980dc7c.png 727w" /></p>
<p><em>NOTE: I used <a href="https://github.com/markwk/qs_ledger">QS Ledger</a> with its <a href="https://github.com/markwk/qs_ledger/tree/master/kindle">Kindle parser and data visualization scripts</a> to get my kindle highlights and generate this report</em>.</p>
<p>While I expected to see a lot more highlights in my Kindle logs, this isn’t what happened. In fact, it was lower than almost any month so far this year.</p>
<h5 id="monthly-books-read-according-to-goodreads">Monthly Books Read, According to Goodreads</h5>
<p><img src="/generated/images/2019-resources/monthly-books-read-716-b78f0a8e0.png" srcset="/generated/images/2019-resources/monthly-books-read-400-b78f0a8e0.png 400w, /generated/images/2019-resources/monthly-books-read-600-b78f0a8e0.png 600w, /generated/images/2019-resources/monthly-books-read-716-b78f0a8e0.png 716w" /></p>
<p><em>NOTE: I used <a href="https://github.com/markwk/qs_ledger">QS Ledger</a> and the <a href="https://github.com/markwk/qs_ledger/tree/master/goodreads">Goodreads integration</a> to collect my books read and reviews and then generate this report.</em></p>
<p>When it comes to books read, I did finish several books over the month, but it was about average for me (roughly a book a week).</p>
<h5 id="initial-conclusions">Initial Conclusions:</h5>
<p>My original hypothesis was that I expected to see some noticeable changes in my media and computer, but from looking at the charts, here is what I found:</p>
<ul>
<li>Computer usage was close to average.</li>
<li>Project time was nearly same too.</li>
<li>Article reading was only a little bit higher than average and aligned with recent months’ growth.</li>
<li>I finished 4 books or about a book a week, which was typical.</li>
</ul>
<p>This meant that my month without YouTube didn’t result in me working more nor was I reading more either (or so it seemed). So what happened?</p>
<h3 id="data-analysis-so-what-changed">Data Analysis: So, what changed?</h3>
<p>The lack of changes in the the areas I expected to change was curious. By my own reckoning I didn’t watch TV or movies much more, and it seemed like I wasn’t reading more either, in spite of having an extra 8 hours a week free now.</p>
<p>When I re-looked at my device usage logs, I noticed something interesting with my iPad usage numbers:</p>
<p><img src="/generated/images/2019-resources/youtube-ipad-usage-trrends-765-22b7b7e2b.png" srcset="/generated/images/2019-resources/youtube-ipad-usage-trrends-400-22b7b7e2b.png 400w, /generated/images/2019-resources/youtube-ipad-usage-trrends-600-22b7b7e2b.png 600w, /generated/images/2019-resources/youtube-ipad-usage-trrends-765-22b7b7e2b.png 765w" /></p>
<p>If you look at the trends we can see a pretty close correlation between iPad usage and YouTube time. This is not surprising since I almost exclusively watch YouTube on my tablet. But when you look at recent weeks AFTER I stopped using YouTube, you’ll notice that my tablet time remained steady and even went up slightly:</p>
<p><img src="/generated/images/2019-resources/ipad-youtube-usage-trends-800-a06f0d821.png" srcset="/generated/images/2019-resources/ipad-youtube-usage-trends-400-a06f0d821.png 400w, /generated/images/2019-resources/ipad-youtube-usage-trends-600-a06f0d821.png 600w, /generated/images/2019-resources/ipad-youtube-usage-trends-800-a06f0d821.png 800w, /generated/images/2019-resources/ipad-youtube-usage-trends-843-a06f0d821.png 843w" /></p>
<p>This is where more detailed screentime logs would be really helpful. Unfortunately, on an iOS device, there is no simple way to get detailed logs of time usage in the actual applications. Right now I just manually log the total time. So, to tease out my recent changes, the best we could do was look at a recent week.</p>
<p>In Screentime, here is what I found:</p>
<ul>
<li>Files 9h29</li>
<li>Instapaper 3h 49</li>
<li>Safari 3h 9m</li>
<li>Spotify 1h 8m</li>
</ul>
<p>This is our first clue. It is finally becoming clear that my YouTube time on the tablet was replaced by reading PDFs. <strong>I wasn’t reading more books or more online articles; I was reading more PDFs!</strong></p>
<h5 id="sidebar-tracking-pdf-reading">SIDEBAR: Tracking PDF Reading</h5>
<p>As far as I know, there is no perfect or complete way to track your PDF reading. I’ve got two options I can use to tease out a bit more intel on my PDF reading changes.</p>
<p>The first way is a <strong>PDF tracking hack</strong> I came that uses a bash script to keep a daily log of changes in the directories where I store PDFs. You can find and try out the code here: https://gist.github.com/markwk/bd9e0b109110a943d32548291e91a241. The main focus is on tagged PDFs, and it does provide a good starting point. Right now the script is collecting various data, but I have yet to write any code to do the needed analysis. Unfortunately, a full data analysis is outside the scope of this experiment too.</p>
<p>The second way to track my PDFs involves a little workaround using how I read and process PDFs. For me, I mostly read PDFs on my Tablet with the Preview App, where I make various highlights. Once I’m finished I tag the PDF in Green, and then, on my Mac, I can use the <a href="https://skim-app.sourceforge.io/">Skim App</a> to export those highlights.</p>
<p>After exporting the highlights, I append the file’s date time. This results in a directory with files that look like this:</p>
<p><img src="/generated/images/2019-resources/pdf-highlights-exports-800-baeb36430.jpg" srcset="/generated/images/2019-resources/pdf-highlights-exports-400-baeb36430.jpg 400w, /generated/images/2019-resources/pdf-highlights-exports-600-baeb36430.jpg 600w, /generated/images/2019-resources/pdf-highlights-exports-800-baeb36430.jpg 800w" /></p>
<p>With a bit of command line parsing, it isn’t hard to count the file numbers by date.</p>
<h5 id="tracking-pdf-reading-according-to-exported-highlights">Tracking PDF Reading According to Exported Highlights</h5>
<p>Here’s a portrait of total PDFs read in terms of the PDF files where I exported highlights:</p>
<p><img src="/generated/images/2019-resources/monthly-exported-highlights-800-4cd112ae2.png" srcset="/generated/images/2019-resources/monthly-exported-highlights-400-4cd112ae2.png 400w, /generated/images/2019-resources/monthly-exported-highlights-600-4cd112ae2.png 600w, /generated/images/2019-resources/monthly-exported-highlights-800-4cd112ae2.png 800w, /generated/images/2019-resources/monthly-exported-highlights-902-4cd112ae2.png 902w" /></p>
<p>This obviously isn’t an exact representation of my PDF reading. I’ll need to use other tracking data at another point to get a better number. But it still shows what we were looking for: <strong>a noticeable increase in PDFs that were exported in July</strong>.</p>
<p>This is the second clue into how my behavior changed without YouTube in the last month: I read more PDFs and exported more highlights.</p>
<h5 id="more-smart-notes">More Smart Notes</h5>
<p>The third datapoint and clue came from looking at my smart notes. Awhile back I changed <a href="http://www.markwk.com/plain-text-life.html">how I take notes</a>. The key aspect is about having a productivity and <a href="http://www.markwk.com/smart-notes.html">knowledge management system</a> I can depend on in my learning and creative endeavors. There are a lot of technical ways you can set up this kind system. I use plain text files since it makes it easy to <a href="http://www.markwk.com/track-writings.html">track my notes</a> too, but a tool like Evernote and many others could be tweaked for similar usage.</p>
<p>Looking at my high-level note logs, we see another change: I took more notes!</p>
<p><img src="/generated/images/2019-resources/smart-notes-count-732-c213210fa.png" srcset="/generated/images/2019-resources/smart-notes-count-400-c213210fa.png 400w, /generated/images/2019-resources/smart-notes-count-600-c213210fa.png 600w, /generated/images/2019-resources/smart-notes-count-732-c213210fa.png 732w" /></p>
<p><strong>So, while I didn’t spend that much more time on my computer or on projects, less YouTube time did translate into more time spent on creating smart notes which were largely based on my increased PDF reading.</strong></p>
<p>Even though I vaguely knew I was reading more PDFs in the last month, it wasn’t until I dug into my logs that I was able to calculate the difference. By my rough ballpark estimate, I spent about 35 to 40 some hours reading PDFs on my tablet. A bit more than normal but not that surprising consider I wasn’t watching YouTube and I wasn’t reading more on my Kindle.</p>
<p>Let’s jump into a few of my conclusions.</p>
<h3 id="conclusion-better-without-it-than-with-it">Conclusion: Better Without It Than With It?</h3>
<p>Since most habits take more than a month to develop, 30-day challenges are a special kind of <a href="http://www.markwk.com/category/science-of-goals/">goal</a> that should aim for discovery and self-understanding. They exist to help you to learn something new or to figure out whether a new habit is right for you. If that short-term change fits your needs, you can work on developing it more over the coming months by converting it into a bigger goal. You might even use something like a <a href="http://www.markwk.com/goal-tracking-with-airtable.html">goal tracker</a> to help you get get there.</p>
<p>In my opinion, the end point of most bigger goals should be about developing new routines. Good habits are one of of the best investments of your time, especially healthy, productive and creative ones. This is due to how they create a permanent and automatic way of life you can use forever. It’s also why bad habits can be so detrimental too. Even though goals are a great thing, self-improvement should not be a forever endeavor. Instead, forget about willing yourself to better and focus on goals that build habits and routines that make good living easy.</p>
<p>There are a lot of great options for month-long challenges, like daily exercise, a gratitude journal, giving up social media, going caffeine free, or working on a hobby among many other. I’m admittedly partial to tracking experiments, so if you’ve never tried tracking, give self-tracking a try for a month and see what you can discover and learn. Check out one of my <a href="http://www.markwk.com/category/track-everything/">tracking guides</a> if you need any tips.</p>
<p>For my recent challenge, I decided to go a month without YouTube. I’m not sure if I classify as a heavy or average user, but an hour a day seemed a bit much for me. Especially when you think about it as nearly 5% of a 168 hour week or nearly 8% of my awake time.</p>
<p>As a long-time self-tracker and someone actively trying to use technology towards self-improvement, this month without YouTube made me realize a few things. <strong>For me, one of the biggest lessons I realized from this experiment was that I can live without YouTube. In fact, it was nice without it.</strong></p>
<p>In many ways YouTube provides an incredible service: endless and largely free videos anytime anywhere I want them. They are also really good at delivering mostly good stuff but with variable returns of really good content. According to neuroscience, this variability of reward makes it highly addictive through how it modulates our dopamine levels among other factors. I was, for lack of a better word, addicted to YouTube.</p>
<p>When I first started my month without YouTube, I found in myself weird automatic tendencies. There were moments where I felt an urge to watch some YouTube. It was a habit that got triggered in certain situations in my day. It’s clear that technologies like YouTube can be effective at changing us. It had changed me. I now some of these triggers and have effectively put in alternative behaviors.</p>
<p>After a month without it, I’m glad that the urge for YouTube is gone. It’s good to know that this kind of addiction wasn’t permanent. All in all, I most missed the comedy videos, and I haven’t found a good way to replace those. Being away from the news cycle was fine for awhile, and something I should be willing to do more regularly. On the educational side, it was hard at a few points where I knew a YouTube video would give me the knowledge I needed and not having access to it could be frustrating. Fortunately, there are plenty of books and articles too on nearly any topic, so educationally lack wasn’t that bad.</p>
<p><strong>So, will I be back on YouTube watching anytime soon?</strong></p>
<p>For now, I don’t plan to immediately start using YouTube again. Overall, I found better ways to spend my time without the time suck and addictiveness of YouTube.</p>
<p>Ultimately, the main question this challenge left me with was this: <strong>Can we find balance in our usage of certain technologies? Or we stuck with all or nothing?</strong></p>
<p>There are a lot of discussions around how addicted we are to our phones, social media and TV. Some even link increased phone usage with depression, social unease, and worsening health, especially in the so-called iGen Generation (Twenge, 2017). Along with social media apps and Netflix, YouTube owns a key portion of many people’s time and attention.</p>
<p>Time limits and logs are a great addition in the technology space. It’s great that both YouTube and my device provide these stats. These numbers can help you get a grip on your usage. Unfortunately, while I don’t have data to back this up, I suspect that the current implementation of time limits fails for many like me. I believe this is due to how they act as warning but fail to deploy richer cognitive strategies for behavioral change (Carey, 2018; Michie, 2013). Technology companies doesn’t actually want to help us live better with technology; they merely feel obliged to make an altruistic gesture.</p>
<p>When it comes to YouTube, if I could limit it to a pure educational tool or better tweak it, I might switch back to it more whole-heartedly. I’d like to be nudged off of using it sometimes. I’d like more control and less feeling entrapped somehow. I’d like it to be tailored to my digital life objectives. For example, I’d like to be able to watch one video without being sucked into ten.</p>
<p>Ultimately, what I’d like to see one day are apps and services that allow us to make the adjustments needed for managing healthier usage. Too many apps and services are dedicated to and designed for capturing your full attention for as long as possible. As a technologist and app developer, I realize much of this is due to the business models behind these “free” services. I would love to have apps that provide value and goodness for a period of my choosing but without ensnaring us for as long as they can keep us. I want tools that align to a conscious usage, rather than some kind of debilitating enslavement.</p>
<p>I know it’s it’s a pipe dream, and until we get there, it seems there were always be trade-offs in the technologies we incorporate into our lives. Sometimes we are better off a service than on it. In the case of YouTube, I think I’m off, off.</p>
<p>What about you? What technologies are you wondering if you are better off of than on?</p>
<p><br /></p>
<h5 id="references">References:</h5>
<ul>
<li>Carey, R. N., Connell, L. E., Johnston, M., Rothman, A. J., de Bruin, M., Kelly, M. P. et al. (2018). Behavior change techniques and their mechanisms of action: a synthesis of links described in published intervention literature. <em>Annals of Behavioral Medicine</em>.</li>
<li>Gollwitzer, P. M. (1999). Implementation intentions: strong effects of simple plans. American psychologist, 54(7), 493. Retrieved from http://kops.uni-konstanz.de/bitstream/handle/123456789/10101/99Goll_ImpInt.pdf?sequence=1&isAllowed=y</li>
<li>Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in experimental social psychology, 38, 69-119.</li>
<li>Michie, S., Richardson, M., Johnston, M., Abraham, C., Francis, J., Hardeman, W. et al. (2013). The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. <em>Annals of behavioral medicine</em>, <em>46</em>(1), 81-95.</li>
<li>Twenge, J. M. (2017). Have smartphones destroyed a generation. The Atlantic, 3. Available online at https://www.theatlantic.com/magazine/archive/2017/09/has-the-smartphone-destroyed-a-generation/534198/.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Over the last 6 months or so, I’ve been tracking my Youtube usage. I came to discover that I was spending on average about an hour a day watching YouTube videos. While there are a lot of things I love about YouTube, it comes with some negatives. So, for one month, I gave up YouTube.Can Meditation Improve Your Attention? Self-Experiment into Mindfulness and Cognitive Testing2019-07-16T00:00:00-07:002019-07-16T00:00:00-07:00http://www.markwk.com/meditation-effect-attention-cognition<p>The reported benefits of meditation are quite impressive: better physical health and happiness, less stress and anxiety, and even improved mental health and cognitive functions.</p>
<p>Unfortunately, I’m skeptical guy. I like data and research to back up such claims. Frankly, the advantages of meditation almost seem too good to be true. Is this a pseudoscience? Can meditation really improve our bodies and our minds? Which types are effective and how?</p>
<p>I recently bought the Muse Brain Sensing Headband in an attempt to ponder some of these questions about the mind and meditation. Muse is a wearable that tracks your brainwaves using EEG and provides real-time neurofeedback while you meditate. It’s relatively popular in the <a href="http://www.markwk.com/quantified-self-mind-map.html">quantified self</a> and biohacker space. Built using dry electrodes and seven sensors, it’s essentially a stripped down, four-channel EEG monitor (compared to standard ten-channel). The aim of the device is to quantify your mental states and subtly train you to meditate better through auditory cues (chirping birds and nature sounds).</p>
<p>In order to explore this claim about the cognitive benefits of meditation, I decided to do a self-experiment, also known as a n=1 or n-of-1 trial. For a few weeks, I took daily cognitive tests both before and after I either meditated with Muse for 10 minutes or did some other activity for roughly the same amount of time. Using neuropsychological tests used to measure cognitive impairment, the test results should be able to tell me if meditating provides any cognitive benefit in areas like attention, information processing, and reaction time.</p>
<p>An <strong>n-of-1 trial</strong> is an experiment done on a single person using a series of interventions over a period of time. Basically it’s a modification of the classic crossover design wherein the invervention and placebo are tested in alternating patterns. The interventions themselves are often blinded (meaning you don’t know which medication you are taking at one time), and they are either completely randomized or given in a balanced cross-over schedule (like ABBABAAB). Outcome measurements are taken throughout to record the effects. These could be biomarkers, tests, surveys or something else. After a few rounds, statistical analysis is used on the outcome measurements to see if there was an effect and to determine which intervention was best. Additionally, you typically note any adverse effects.</p>
<p>In my own case related to meditation effect, the outcome I wanted to measure was an improvement in my cognition. The intervention or experimental variable I was testing was meditation, compared to other activities, which would be the placebo. It was impossible to blind the intervention, since you obviously know if you are meditating or not, so you might call it an open label experiment.</p>
<p>The tests I used were taken from standard neuropsychological assessments, frequently used to diagnosis cognitive impairment and in clinical trials on different drugs and supplements. I took the tests in the same place and around the same time daily, shortly after breakfast and roughly one hour after waking up. After several weeks, I then did some statistical and data analysis to check and visualize the effects and to see if they were significant.</p>
<p><br /></p>
<p>So, what were the results?</p>
<p><strong>Meditation improved my attention and other cognitive functions! Unfortunately so did any other activity I did. In fact, the main takeaway from the final results showed that meditation was not the most significant variable on cognitive improvements. Instead, the experiment revealed that simply re-taking the tests led to both an intraday improvement (meaning an improvement in the before and after) and a cumulative, linear improvement (meaning my first 5 to 10 scores were lower than my last 5-10 scores). In short, what I found was my cognitive testing was improved mostly through a combination of practice and training effects. I got better not because I meditated, but because I tested again and again.</strong></p>
<p>In the rest of this post, I want to look at this experiment and the results in a bit more detail and explain what happened and why. Hopefully by the end of the post you’ll understand what are cognitive tests, a bit more about meditation, and how to do your own self-experiment on the meditation effect!</p>
<!--more-->
<hr />
<p><strong>A “TRY THIS AT HOME” WARNING NOTE</strong>: My account and approach to meditation described here should largely be safe for most anyone to try. That said, meditation can be an intense experience with a history of adverse effects (Love, 2018; Kornfield, 1979; Lindahl, 2017). Most meditators, especially 10-minutes-a-day types, should be fine.</p>
<p><strong>“Citizen Science” Code and Data Analysis</strong>: Please do tell me how your meditation effect experiment goes. For the data analysis part, the <a href="https://github.com/markwk/qs_ledger/blob/master/bonus_data_analysis_notebooks/Quantified-Mind-Com-Data-Analysis.ipynb">code</a> is available for free with QS Ledger, an open source tool for Quantified Self Personal Data Aggregation and Data Analysis. If you want a quick data analysis, send me your testing results by <a href="http://www.markwk.com/contact/">email</a>, and I’d be happy to generate a PDF report for you!</p>
<hr />
<h3 id="my-n1-self-experiment-protocol-how-to-test-the-effect-of-meditation-on-attention">My N=1, Self-Experiment Protocol: How to Test the Effect of Meditation on Attention</h3>
<p>The short version of what I did: Over the span of 14 days I took daily cognitive test using Quantifie-Mind.com both before and after I either meditated or did some other activity for roughly 10 minutes. I then did some data analysis to examine at the effects.</p>
<p>Below is a description in greater detail. (Apologies for the pseudo-academic tone. Been reading a bit too much scientific literature lately.)</p>
<h5 id="methods-and-trial-design">Methods and Trial design:</h5>
<p>For this experiment, I used an online testing platform called Quantified Mind. The tests were Go / No-Go and Cued Attention, which focus mostly on visual attention and information processing as well as a degree of executive function, in form of inhibition control. I followed Quantified Mind’s meditation experiment protocol, which recommended that on alternating days I do the tests both before and after either meditating or an alternative activity. I took the tests each morning over the span of roughly two weeks.</p>
<h5 id="sidebar-what-is-meditation">SIDEBAR: What is meditation?</h5>
<p><img src="/generated/images/2019-resources/what-is-meditation-mini-356-1f6b2592c.jpg" srcset="/generated/images/2019-resources/what-is-meditation-mini-356-1f6b2592c.jpg 356w" /></p>
<p>In <a href="http://www.markwk.com/what-is-meditation.html">What am I meditating for? In Pursuit of A Definition of Meditation</a>, I attempted to tackle the question of what is meditation. While a lot features and factors can be used to define meditation and the different types, the best definition I’ve been able to come up with is that meditation is:</p>
<blockquote>
<p>A multi-step process whose two principal components are 1. the methods or cognitive strategies used and 2. the enhanced mental states it brings about.</p>
</blockquote>
<p>Overall, meditation involves several stages and rituals, but the end result is to bring you to a certain meditative state.</p>
<p>What I mean by this definition then is that different meditation types use different cognitive strategies (like concentrating on breath, counting breathes, open monitoring, etc.) in order to bring you towards a certain enhanced mental state. So, if you are trying classify meditation types, among other things, it is best to specify the methods used and target meditative or mental state achieved.</p>
<h5 id="intervention-10-minutes-of-meditation-vs-another-activity">Intervention: 10 Minutes of Meditation vs. Another Activity</h5>
<p>The main variable I was trying to test for was the effect of meditation. For my meditation practice, I used Muse, an EEG wearable monitor and neurofeedback meditation app. After putting on the device and syncing it with my phone, I did 10 minutes of neurofeedback mindfulness meditation, which is a form of concentration meditation where I attended to my breath and received auditory feedback on how calm I was.</p>
<p>A full dive into Muse and Neurofeedback-support meditation is outside the scope of this post, but to be more precise, based on our definition of meditation, here are the key characteristics:</p>
<ul>
<li><strong>Method</strong>: Mindfulness-focused concentrative meditation involving attention to breathing and supported by EEG neurofeedback based on calm mental state.</li>
<li><strong>Enhanced Cognitive State</strong>: The intended mental state was a calm, relaxed state with attentive concentration on breathing.</li>
</ul>
<p>For the active control, I did a few different activities, including reading and sorting computer files.</p>
<h5 id="sidebar-neuropsychological-and-cognitive-testing-can-we-quantify-cognitive-functions-with-games-and-quizzes">SIDEBAR: Neuropsychological and Cognitive Testing: Can we quantify cognitive functions with games and quizzes?</h5>
<p><img src="/generated/images/2019-resources/reaction-time-test-example-800-1d88d13f2.jpg" srcset="/generated/images/2019-resources/reaction-time-test-example-400-1d88d13f2.jpg 400w, /generated/images/2019-resources/reaction-time-test-example-600-1d88d13f2.jpg 600w, /generated/images/2019-resources/reaction-time-test-example-800-1d88d13f2.jpg 800w, /generated/images/2019-resources/reaction-time-test-example-1000-1d88d13f2.jpg 1000w" /></p>
<p><em>Screenshot showing a reaction time test from Quantified-Mind.com</em></p>
<p><br /></p>
<p>All good experiments require a outcome measurement. This lets us know if a drug, intervention or some other change had an effect compared to the control. For our purposes, cognitive tests or neuropsychological assessments provide the real “litmus test” as to whether meditation any affect our attentional capability. Cognitive testing, which pre-dates much of today’s neuroimaging tools, remains a reliable method to study cognitive functions, like memory and executive control.</p>
<p>Used somewhat loosely and still ill-defined in much of the literature, <em>cogntive functions</em> describe the capacities of our brains in what they are able to accomplish. While language, social cognition and perceptual and motor functions are also often mentioned, the three most commonly cited cognitive functions are:</p>
<ol>
<li><strong>Attention</strong>: Attention is a behavioral and cognitive process of selective concentrating on certain information while ignoring others. Attention, concentration or focus describes a cognitive function that is able to undertake goal-directed behavior amongst the plethora of sensory stimuli originated from our environment and minds. Attention is a state of arousal, which we might colloqually call our energy level. This means that our ability to pay attention depends on BOTH being awake and ready AND able to selective focus on certain inputs over others.</li>
<li><strong>Memory</strong>: Often viewed a unitary function, memory involves multiple capacities and interconnected ability. Broadly speaking, memory describes our ability to retain information. In technical sense, memory as a cognitive function refers to the ability of our brains to encode, store, and retrieve information when needed. This includes among other things our short-term or working memory.</li>
<li><strong>Executive Functions</strong>: Often referred to as our “higher” cognitive functions, executive functions entail planning, reasoning and orchestrated and controlled behavior as well as related aspects like inhibition control, intelligence (especially fluid intelligence), and even creativity.</li>
</ol>
<p>Neuropsychological and cognitive testing describe the various methods devised to quantify and measure our minds. Much of cognitive testing aims to assess impairment. This can include screening for cognitive disorders and brain diseases as well as brain trauma like a concussion. Increasingly cognitive testing has been used to study neuroenhancement and efforts to augment our memory, reaction time and more.</p>
<p>Several dozen, if not hundreds, of cognitive and neuropsychological tests exist to measure different aspects of the human mind. Here are a few common ones and what they test:</p>
<ul>
<li><strong>Reaction Time</strong>: These tests assess how quickly a person can respond to a single stimulus (like a circle that turns from white to green) by pressing a button. Several variations exist, like choice reaction time, Go/No-Go, that require information processing and inhibition control. Reaction time tests are a relatively direct means of measuring cognitive functions like processing speed and various attentional deficits associated with aging, depression and neurological impairment (Lezak, 2004).</li>
<li><strong>Stroop Task</strong>: This is a classic cognitive test focused on executive function. Named after the English psychologist John Ridley Stroop, during the test, participants are asked to attend alternatively to either the written name of the color (e.g. “red”) or noticing the printed color of those words. When the color of the letters making up the name differs from the colour the word describes, participants exhibit a delayed response by participants. The Stroop task has been used to measure frontal lobe dysfunction and aging among other effects (Lezak, 2004).</li>
<li><strong>Digit Span</strong>: This is another classic test used to measure memory and, to some extent, attention. During the test, digits are presented to the participant one by one and then you are ask to repeat them back. There are a few variations including forwards, backgrounds, auditory, and visual as well as some involving repeating back a pattern of filled-in box on a grid. The normal range for Digits Forward is 6 ± 1, and the difference between digits forward and digits reversed tends is a bit above 1.0 (Lezak, 2004).</li>
</ul>
<p>While most cognitive tests strive to isolate and test individual cognitive functions, the realitity is that these tests, like our minds themselves, rarely test a single function. In fact most testing requires a degree of several cognitive abilities for them to be done. For example, we need a degree of attention, executive control and memory to do most of what we do in life. Our adaptive minds are a tight coupling of several abilities and brain regions.</p>
<p>For our own purposes in this experiment, these tests provide the most reliable and objective way to measure any possible acute cognitive effects following meditation, meaning our scores should theoretically increase after meditation.</p>
<h5 id="outcome-measurements-cognitive-tests-on-quantified-mindcom">Outcome Measurements: Cognitive Tests on Quantified-Mind.com</h5>
<p>In order to assess the impact of meditation on my cognition, I took daily cognitive tests before and after. After reviewing a few different tools, the testing platform I elected to use was Quantified-Mind.com. It’s free and provides a pretty wide range of standard and adapted neuropsychological tests, which can be used to test various cognitive abilities.</p>
<p><img src="/generated/images/2019-resources/go-no-go-reaction-time-test-400-3dc0fc5fc.png" srcset="/generated/images/2019-resources/go-no-go-reaction-time-test-400-3dc0fc5fc.png 400w" /></p>
<p>The specific tests used were:</p>
<ul>
<li><strong>Go/No-Go</strong>: In this test you are shown either a black screen with either a white square at the top (target stimuli) or a white square at the bottom (stimuli meant to be ignored). The cards are presented one by one and your task is to tap the space key when the target appears and ignore the non-target. Statistics are kept on how fast you react and your accuracy (successful taps vs. tapping on non-target), and a final test score is generated. This attention test is similar to the T.O.V.A.</li>
<li><strong>Cued Attention (dot)</strong>: This test present you with two boxes one on the left and one of the right. When a dot appears inside of one of the boxes, the task is tap the space bar. Distractions and non-target stimulus are provided by flashing boxes, which you need to ignore. The object is to tap only when the dot appears in the box. Accuracy and reaction time are required, and an aggregate score is calculated at the end. This is another attention test and quite similar to Go/No-Go and is based on the popular Posner spatial cueing task.</li>
</ul>
<p>Both of these tests are primarily intended to quantify your attention and reaction time. They also test to an extent your executive function in the form of inhibitory control and speed of information processing.</p>
<p>While reaction time and accuracy were measured throughout each test, the only final information provided was a composite or aggregated score on each test. So the final outcome measurement used were test results of Go / No-Go and Cued Attention.</p>
<h5 id="sampling-sequence-and-schedule">Sampling, Sequence and Schedule</h5>
<p>In following the protocol from Quantified-Mind.com, I took the tests 28 times over two weeks. I took them in the same space and roughly the same time each morning, following breakfast and a coffee. I followed an alternating daily schedule of meditation on one day (A) and an alternative activity on the other (B). So the pattern was AB repeated. I took the initial tests, then did an activity and retook the tests. This resulted in a 14-day sample of 28 test results.</p>
<h5 id="data-analysis">Data Analysis:</h5>
<p>Unfortunately, at the time of writing, Quantified Mind does not provide actual analysis of the experiment’s test results. It only gives a few generic charts on scoring trend on individual tests over time.</p>
<p>So for that reason, data processing, exploratory statistical analysis and data visualization were all done using Python. I have made public the code I created as part of QS Ledger <a href="https://github.com/markwk/qs_ledger/blob/master/bonus_data_analysis_notebooks/Quantified-Mind-Com-Data-Analysis.ipynb">here</a>. Besides a visual comparison, I used (frequentist) statistical methods to look at effect size based on differences between means using a simple comparison and the more robust Cohen’s <em>d</em>.</p>
<p>Some additional factors (sleep, HRV, exercise, etc.) and analysis on them have been will be provided in a later followup.</p>
<h3 id="results-did-meditation-improve-my-attention">Results: Did Meditation Improve My Attention?</h3>
<h4 id="results-of-exploratory-data-analysis">Results of Exploratory Data Analysis</h4>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-figure-01-800-1dd042232.png" srcset="/generated/images/2019-resources/meditation-effect-on-attention-figure-01-400-1dd042232.png 400w, /generated/images/2019-resources/meditation-effect-on-attention-figure-01-600-1dd042232.png 600w, /generated/images/2019-resources/meditation-effect-on-attention-figure-01-800-1dd042232.png 800w, /generated/images/2019-resources/meditation-effect-on-attention-figure-01-1000-1dd042232.png 1000w" /></p>
<p><em>FIGURE 1: Box-and-Whisker Plot showing an aggregate summary of scores in each test scenario. The most visually obvious improvement is in 1. Before and After Activity (Go/No-Go) and 2. Before/After Meditation (Cued Attention). Is this evidence of cognitive improvement?</em></p>
<p><br /></p>
<p><strong>The results of the experiment did not provide sufficient evidence to support the claim that meditation improves attention and related cognitive functions. By contrast, the strong effect appears to be practice effect (aka “test-retest”).</strong></p>
<p>As summarized in Table 1, the highest scores were for the most part <em>after</em> either meditating or an activity. As showed in Table 2 and explained before, statistical analysis using Cohen’s D did indicate there was a sizable effect in before and after with the highest effect size being 1.08 in before/after meditation on Cued Attention test.</p>
<p><br /></p>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-table-01-800-66a4f393d.jpg" srcset="/generated/images/2019-resources/meditation-effect-on-attention-table-01-400-66a4f393d.jpg 400w, /generated/images/2019-resources/meditation-effect-on-attention-table-01-600-66a4f393d.jpg 600w, /generated/images/2019-resources/meditation-effect-on-attention-table-01-800-66a4f393d.jpg 800w, /generated/images/2019-resources/meditation-effect-on-attention-table-01-1000-66a4f393d.jpg 1000w" /></p>
<p></br></p>
<p>On the surface (as shown in FIGURE 1 and TABLE 2) this does seem to point towards cognitive improvements, especially in the improvement of Cued Attention following meditation. But when looking at FIGURE 2 showing the overall trend and before/after effects, the apparent effect of meditation appears to be more likely attributed to the practice or test-retest effect, rather than related to one of the test variables.</p>
<p>In fact, the main results of this experiment give more evidence to one’s ability to improve cognitive testing scores through practice on testing, rather than improvements caused by an activity like meditation. If we compare the initial five or six days of testing with last five or six days, we clearly see signs of overall improvements on both tests, regardless of meditation effect.</p>
<p><br /></p>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-figure-02-800-05dab86ca.jpg" srcset="/generated/images/2019-resources/meditation-effect-on-attention-figure-02-400-05dab86ca.jpg 400w, /generated/images/2019-resources/meditation-effect-on-attention-figure-02-600-05dab86ca.jpg 600w, /generated/images/2019-resources/meditation-effect-on-attention-figure-02-800-05dab86ca.jpg 800w, /generated/images/2019-resources/meditation-effect-on-attention-figure-02-1000-05dab86ca.jpg 1000w" /></p>
<p><em>FIGURE 2: Scoring trends showing linear improvements in results of both before (blue) and after (orange). This is most indicative of practice effects both on linear scale of the experiment and on intraday testing, i.e. before and after.</em></p>
<p><br /></p>
<h4 id="results-of-statistical-analysis">Results of Statistical Analysis</h4>
<p>Two statistical analysis methods were used.</p>
<p>First, a simple comparison of means was done on the results of the the four scenarios. Results indicated improvements between before and after on both meditation and activity. Here are the top five mean differences:</p>
<ol>
<li>Before/After Meditation: Cued Attention: 41.17</li>
<li>Before Activity vs. Meditation: Cued Attention: 33.6</li>
<li>Before/After Activity: Go / No-Go: 31.58.</li>
<li>Before/After Meditation: Go / No-Go: 19.6</li>
<li>After Activity and Meditation: Go / No-Go: 15.6</li>
</ol>
<p>Second, Cohen’s <em>d</em> effect size was used to look at effect size and explore any potential significance. The results are provided in TABLE 2.</p>
<p>As a quick reminder, according to Cohen’s power primer (1992), d = 0.20 is considered a small effect size, d = 0.50 is a medium effect size, and d = 0.80 is a large effect size. Anything below 0.2 is generally viewed as no effect.</p>
<p>As shown in TABLE 2, the results showed a significant Cohen’s effect size in before and after for both meditation and activity. The highest specific effects were on Cued Attention in Before/After Meditation (1.08) and and Go / No-Go in Before/After Activity (0.50).</p>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-table-02-700-0954015b5.png" srcset="/generated/images/2019-resources/meditation-effect-on-attention-table-02-400-0954015b5.png 400w, /generated/images/2019-resources/meditation-effect-on-attention-table-02-600-0954015b5.png 600w, /generated/images/2019-resources/meditation-effect-on-attention-table-02-700-0954015b5.png 700w" /></p>
<p>Once again this does appear to indicate a strong effect, and it would be easy to interpret the results as an endorsement of meditation as a key to cognitive improvements. But once again we see more the effects from re-testing.</p>
<h4 id="final-results-practice-effect-rather-than-meditation-improve-cognition">Final Results: Practice Effect, rather than Meditation, Improve Cognition</h4>
<p>As shown in TABLE 2, all testing results improved during re-testing. In aggregate of Before/After, Go / No-Go showed a Cohen Effect Size of 0.47 and Cued Attention in 0.68. This would indicate a medium effect overall. Due to the sampling size (28), further testing is likely needed to confirm the results in greater detail and with higher confidence.</p>
<p>While in this experiment, I was hoping to prove that meditation improves cognition, the evidence does not seem to show that. In fact, the strong effect difference appears to be simply re-taking the test shortly after just taking it. Moreover, repeatedly taking the tests over showed cumulative, linear improvements over time (FIGURE 2). In short, this experiment showed the practice effect.</p>
<p>The <strong>practice effect</strong> is a well-known issue in neuropsychological and cognitive testing (Collie, 2003); Wesnes, 2002). It describes the phenomenon where testing results improve simply by retaking a test, regardless of any other changes or interventions. This can lead to the false sense that an intervention causes an improvement, rather than merely following along with improvements gained by retesting. A few reasons have been proposed for why this happens, including greater familiarity with procedure, learning, and even a placebo effect.</p>
<p>In traditional experiments, there are a few ways to overcome the practice effect and linear trends in general. One method is to take different tests that mitigate against practice effects on a repeat taking of the same test. So at the start of the experiment you might take testing battery A and then during the experiment you’d take testing battery B. At the end you would either repeat A or even do C.</p>
<p>Arguably the best method to deal with the practice effect is through experimental design using either parallel groups or a crossover design. Parallel groups allows you to compere the size of the improvements between groups, and, as such, in spite of the practice effects, it lets you see if improvements are greater for the test intervention compared to a control. One problem here can be a placebo effect if you do not provide an active control, meaning a control that appears to also have benefits to cognition. For example, in this experiment you might use a relaxation technique as an active control to meditation.</p>
<p>In contrast, a <strong>crossover design</strong> is the best method in our particular case. Crossover experimental design involves periodic switching between the interventions and the control with the same measurements throughout (Bose, 2009; Lui, 2016). This means you receive both the control and the intervention and have measurements from both situations. Subsequently during analysis you compare this different blocks. There are some issues with crossover designs, especially with drug testing, due to carryover effects that often require washout periods. In most cases though, crossover design provides a reliable solution for dealing with practice effects among other experimental challenges.</p>
<p>There are a few ways to do this but a commonly recommended cross-over schedule looks something like ABBABAAB. This type of design provides a way to capture linear trends and enough of a balance that both control and intervention get tested equally and fairly.</p>
<p>The protocol provide by Quantified Mind could be described as a “naive” crossover design since it simply alternated between A (meditation) and B (alternative activity). This makes sense from an easy of implementation. While not the most optimal, it worked reasonably well for this experiment and provided enough data to show the practice effect. In retrospect a more robust AB/BA design would be my recommendation (For example, ABBABAAB x 4 = 28).</p>
<h4 id="discussion-of-final-results-and-experiment">Discussion of Final Results and Experiment</h4>
<p>Overall, I believe the primary conclusion of this experiment should be that the results pointed to a limited or null effect of meditation, at least compared to either an alternative activity or simply re-test, practice effects. As such, evidence does not point towards meditation as an acute means to improving subsequent cognition tests.</p>
<p>Due to the limited number of samples (28) and short period (14 days), these conclusions should not be viewed as definitive proof on any of the core questions we hoped to answer, most notably: Does Meditation Improve Your Attention?</p>
<p>I believe these initial results offer a good starting point for both more research and future experiments that includes more testing samples, differerent meditation types, and a more robust crossover design. Ideally, along with more samples, a wider range of activities and meditations can offer clues into acute effects of pre-testing situations and activities. Our question then becomes: What activities best cue or prime us towards higher performance for cognition as measured by cognitive tests?</p>
<h3 id="conclusion-meditating-for-mental-states-or-neuroenhancement">Conclusion: Meditating for Mental States or Neuroenhancement?</h3>
<p>Brain studies and how to track my mind have been two of my learning goals this year. I bought the Muse EEG brainwave monitor and neurofeedback meditation tool as a way to explore these interests, and I plan to dig deeper into this topic more this year.</p>
<p>Meditation is an interesting test case for neuroenhancement since it is non-pharmaceutical and there appears to be a long history of humans believing there are benefits to meditating. I decided meditation deserved further study, especially if there was a way to quantify it using a device like Muse. This experiment on cognitive testing and meditation provided a good starting point to test a few questions and to explore a number of interrelated topics related to the mind.</p>
<p>Going into the experiment and based on my previous meditation experiences, my “naive” working hypothesis was that meditation would improve my test scores on attention. It seemed obvious that meditation as a method to bring you more in-tune with your mind was a way to level up the brain and it should be recognizable through neuropsychological testing, especially those looking at attention and reaction time. I didn’t expect huge effects, but I expected some. I even initially typed an introduction to this post under that assumption.</p>
<p>Unfortunately, so far, I was not able to prove significant cognitive benefits from meditation alone during my n=1 crossover experiment comparing meditation and an alternative activity by measuring cognitive testing results. Instead, what I found was that cognitive testing as such (meaning repeat testing) and done over time were the largest factors in my testing improvements. By taking the tests repeatedly and over time lead to the biggest improvement, not meditation.</p>
<p>On the one hand, we might interpret the results of this experiment into the cognitive effects of meditation as rather disappointing. Initially I was disappointed too. After all of this time meditating, it seems it wasn’t bringing any acute, transferrable benefits to my cognitive performance. But maybe we shouldn’t be so quick to discount the benefits of meditation. For one thing, this experiment did not test long-term effects nor the question of dosage. I can’t say if the linear improvements were due completely to re-testing and practice or if a small part came from my meditation. I suspect meditation didn’t change anything since changing our brains likely requires more than an hour or two of meditative practice.</p>
<p>On the other hand, my cognitive testing improvements are a positive and interesting result, since it shows we can improve our minds in a way. Whether this is the practice effect or a training effect is impossible to determine in this particular experiment. It might indicate the possibility that so-called “brain games” and cognitive training are a way to improve our minds. Unfortunately, the research is a bit mixed on the benefits. There is research showing training does improve cognitive testing results, but most also show that the effects don’t transfer. What this means is an improvement in certain cognitive tests don’t result in an overall improvement in other areas. In general I’m not planning to invest my time into brain training, but in view of some cases where the effects were transferrable, especially in memory, it isn’t something we should completely count out.</p>
<p>Besides a personal verification of the practice and training effects, three observations stood out to me from this experiment.</p>
<p>First, we probably shouldn’t expect acute cognitive benefits from meditation. In fact, meditation may actually slow our cognition in some ways. While I expected the opposite, this result is not surprising when you think that in most meditation practices you close your eyes and center your attention on yourself. This would mean a shifting of cognitive resources away from situational and sensory awareness to certain inner watchfulness. One good fatham meditation hinders rather than enhances certain visual attentive abilities. For example, I’d be interested to see how other forms of meditation, especially open-eyed and open monitoring, might offer potentially different results on my attention testing.</p>
<p>Second, this might come as a surprise, but, in spite of the negative results on the effect of meditation on my cognition, I plan to continue meditating. For one thing more testing and research is needed on neuroenhancement potential with meditation as well as the other reported health benefits. But, to be honest, as I realized during writing my post on <a href="http://www.markwk.com/what-is-meditation.html">What is Meditation?</a>, meditation isn’t just about the benefits. My definition of meditation as a multi-step process of methods and cognitive states intentionally leaves out anything to do with benefits. Instead, meditation has historically been and remains a process about reaching and exploring different mental states. For the purpose of exploring and understanding our minds, meditation is one of the most amazing phenomenological tools we have.</p>
<p>Meditation is one of the most robust human traditions ever assembled of humans monitoring their minds, literally minds “minding” minds. Interestingly, meditation in the Buddhist tradition can be understood to some extent as one’s personal striving to cultivate a mind capable of self- and universal understanding. Leaving aside the religious and belief claim, it’s not all that surprising how often the study of the mind and brain overlaps with various meditative practices. Neurologists and researchers have even used Buddhist meditators and texts as phenomenological guides into different mental states and consciousness (Depraz, 2003; Varela, 2017, orig 1992). For that reason among others, I plan to continue meditating.</p>
<p>Third, when it comes to the question of neuroenhancement and meditation, dosage is a pretty substantial one. In fact, like the question of what is meditation, I find too much of the academic literature on meditation and its benefits ignores or fails to answer the pointed questions of <em>how much</em> and <em>how long</em>. Practically-speaking how much do we need to meditate to expect a health or cognitive benefit? Is a little bit daily enough or do we need to invest in a more substantial practice? What type of meditation is most effective toward certain goals? Furthermore, can technology, specifically neurofeedback, entrain your brain to reach certain states faster and shorten the time need to get to meditation-induced brain changes?</p>
<p>Frankly, my current experiment wasn’t designed to answer these questions. A deeper exploration is needed into EEG, brainwaves and neurofeedback in general as well as further testing.</p>
<p>While meditation might be one way, for now, I think we are best sticking with the more well-established routes to cognitive enhancement: exercise, good health, music playing, sleep, learning foreign languages and new skills, and possibly even nootropics.</p>
<p><br /></p>
<p>Best of luck and happy tracking!</p>
<p><br /></p>
<h5 id="references">References:</h5>
<ul>
<li>Bose, M., & Dey, A. (2009). <em>Optimal Crossover Designs</em>. World Scientific.</li>
<li>Cohen, J. (1992). A power primer. Psychological bulletin.</li>
<li>Collie, A., Maruff, P., & Darby…, D. G. (2003). The effects of practice on the cognitive test performance of neurologically normal individuals assessed at brief test–retest intervals. Journal of Clinical and Experimental Neuropsychology.</li>
<li>Depraz, N., Varela, F. J., & Vermersch, P. (2003). On Becoming Aware. John Benjamins Publishing.</li>
<li>Hodges, J. R. (2017). Cognitive Assessment for Clinicians. Oxford University Press.</li>
<li>Kornfield, J. (1979). Intensive insight meditation: A phenomenological study. The Journal of Transpersonal Psychology, 11(1), 41.</li>
<li>Lezak, M. D., Howieson, D. B., Loring, D. W., Fischer, J. S., & others. (2004). Neuropsychological Assessment. Oxford University Press, USA.</li>
<li>Lindahl, J. R., Fisher, N. E., Cooper, D. J., Rosen, R. K., & Britton, W. B. (2017). The varieties of contemplative experience: A mixed-methods study of meditation-related challenges in Western Buddhists. PloS one, 12(5).</li>
<li>Lui, K.-J. (2016). <em>Crossover Designs</em>. John Wiley & Sons.</li>
<li>Love, Shayla (2018). Meditation Is a Powerful Mental Tool—and For Some People It Goes Terribly Wrong. Vice.com. Available at https://www.vice.com/en_us/article/vbaedd/meditation-is-a-powerful-mental-tool-and-for-some-it-goes-terribly-wrong.</li>
<li>More, L., Lauterborn, J. C., Papaleo, F., & Brambilla, R. (2019). Enhancing cognition through pharmacological and environmental interventions: Examples from preclinical models of neurodevelopmental disorders. Neuroscience \& Biobehavioral Reviews.</li>
<li>Nash, J. D., & Newberg, A. (2013). Toward a unifying taxonomy and definition for meditation. Frontiers in psychology, 4, 806.</li>
<li>Ospina, M. B. (2008). Meditation Practices for Health. DIANE Publishing.</li>
<li>Posner, M. I. (1980). Orienting of attention. Quarterly journal of experimental psychology, 32(1), 3-25.</li>
<li>Varela, F. J., Thompson, E., & Rosch, E. (2017, orig 1992). The Embodied Mind. MIT Press.</li>
<li>Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A. et al. (2018). Mind the hype: A critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science, 13(1), 36-61.</li>
<li>Wesnes, K., & Pincock, C. (2002). Practice effects on cognitive tasks: a major problem? <em>The Lancet Neurology</em>, <em>1</em>(8), 473.</li>
</ul>
<p><br /></p>
<h3 id="appendix-of-additional-figures">Appendix of Additional Figures</h3>
<h5 id="figure-3-raw-scoring-trend-throughout-experiment">Figure 3: Raw Scoring Trend Throughout Experiment</h5>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-figure-03-800-0a30291c9.jpg" srcset="/generated/images/2019-resources/meditation-effect-on-attention-figure-03-400-0a30291c9.jpg 400w, /generated/images/2019-resources/meditation-effect-on-attention-figure-03-600-0a30291c9.jpg 600w, /generated/images/2019-resources/meditation-effect-on-attention-figure-03-800-0a30291c9.jpg 800w, /generated/images/2019-resources/meditation-effect-on-attention-figure-03-1000-0a30291c9.jpg 1000w" /></p>
<p><br /></p>
<h5 id="meditation-metrics-according-to-muse">Meditation Metrics According to Muse</h5>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-table-03-800-71447e5e0.jpg" srcset="/generated/images/2019-resources/meditation-effect-on-attention-table-03-400-71447e5e0.jpg 400w, /generated/images/2019-resources/meditation-effect-on-attention-table-03-600-71447e5e0.jpg 600w, /generated/images/2019-resources/meditation-effect-on-attention-table-03-800-71447e5e0.jpg 800w" /></p>
<p><img src="/generated/images/2019-resources/meditation-effect-on-attention-figure-04-800-3c0909401.jpg" srcset="/generated/images/2019-resources/meditation-effect-on-attention-figure-04-400-3c0909401.jpg 400w, /generated/images/2019-resources/meditation-effect-on-attention-figure-04-600-3c0909401.jpg 600w, /generated/images/2019-resources/meditation-effect-on-attention-figure-04-800-3c0909401.jpg 800w" /></p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}The reported benefits of meditation are quite impressive: better physical health and happiness, less stress and anxiety, and even improved mental health and cognitive functions.Scoring Your Weekly Goals2019-07-15T00:00:00-07:002019-07-15T00:00:00-07:00http://www.markwk.com/scoring-goals<p>I leverage science, goals and goal-setting in a number a ways. But recently, I started something new. I started “scoring” them. This has proven a good way to periodically check my efforts over time and my organization in reaching the things that matter.</p>
<p>When it comes to goals, one way I use them is to prioritize what’s important.I use <a href="http://www.markwk.com/multistage-goal-pursuits.html">goals as multi-step pursuits</a>. I’m also a believer in the <a href="http://www.markwk.com/science-of-goals.html">science of goal-setting</a> to improve my chances of reaching epic objectives. I use goal planning as a method to balance out a number of competing interests and objectives over time. For longer-term stuff, I even built a <a href="http://www.markwk.com/goal-tracking-with-airtable.html">DIY goal tracker</a> to help set, schedule, and align tasks and projects (aka sub-goals) towards reaching my yearly and lifetime “missions.”</p>
<p>Unfortunately one of the challenges with goals is how to quantify them. Over the past several months, I started to keep score on goals on my weekly goals. When combined with constructive <a href="#">goal setting</a>, I’ve found it to be a good way to ensure I’m <em>goal-aligned</em> in my doings.</p>
<!--more-->
<p>The basic idea is simple: <strong>Each week (or month or even day, if that’s your thing) write down a list of your goals. Then at the end of that period during your weekly review, mark how how many were completed and then calculate a score based on goals completed and those left undone.</strong></p>
<p>The exact scoring isn’t all that important. Just strive to be consistent. You could simply divide total completed over total set, and you’ll get a completion percentage. The higher percentage could be used to quantify a better week in pursuit of goal completion.</p>
<p>For reference, here is how I score mine: Each week I set one or two priority goals, which I call “Weekly Things,” and also list another 4-8 smaller goals, which I call “Support Objectives.” I allot 3 possible points to the weekly things. So if I get 1 out 2 done, then I get 1.5 points. If I get 2 out of 2, then it’s three points. For the support objectives, I also allot 3 points and simply divide total completed over total tried and multiple by 3.</p>
<p>The end result is my weekly score goal, which in my case has ranged from a low 2 to a high 4 with the occasional 5. Obviously it’s technically possible to get 6 points, but that’s not my intention here. If I’m scoring that high, it likely because I didn’t set ambitious enough goals. In general, I aim for anything over 4.</p>
<p>Obviously, this method requires you to consistently set and manage your goals, which, to be honest, if you aren’t doing that, start with that. For example, do a weekly review. I’ve written two guides to help get you started: <a href="http://www.markwk.com/2016/01/power-of-weekly-review.html">Power of Weekly Reviews</a> and <a href="http://www.markwk.com/data-driven-weekly-review.html">Data-Driven Reviews</a>.</p>
<p>Beyond a basic goal list and some form of scoring, there are a few other things to consider. Personally, I find it good to have a balance of different types of goals, meaning work, health/fitness, learning, personal and creativity goals. Aim to prioritize your list. If you are just getting started, start with a few items and build momentum. I find it effective to combine goal setting with a bit of task setting in my <a href="http://www.markwk.com/task-tracking-with-todoist.html">task tracker</a>. That way I’ve blocked out specific tasks and time slots in my calendar for my goals.</p>
<p>Here’s a chart of my latest numbers:</p>
<p><img src="/generated/images/2019-resources/goal-scoring-chart-754-80ae80ff0.png" srcset="/generated/images/2019-resources/goal-scoring-chart-400-80ae80ff0.png 400w, /generated/images/2019-resources/goal-scoring-chart-600-80ae80ff0.png 600w, /generated/images/2019-resources/goal-scoring-chart-754-80ae80ff0.png 754w" /></p>
<p>Unfortunately, as you can see, I haven’t been as successful in completing my goals in late May and into June (Weeks 20 to 24), but recently I’m steadily getting back on track!</p>
<p><br /></p>
<p>Best of luck with your goals and happy tracking!</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}I leverage science, goals and goal-setting in a number a ways. But recently, I started something new. I started “scoring” them. This has proven a good way to periodically check my efforts over time and my organization in reaching the things that matter.What am I meditating for? In Pursuit of A Definition of Meditation2019-07-10T00:00:00-07:002019-07-10T00:00:00-07:00http://www.markwk.com/what-is-meditation<p>Meditation is one of the most touted habits we are all <em>supposed</em> to do these days. Meditation and mindfulness are lauded for a range of associated benefits from physical and mental health to cognitive improvements and beyond. It’s believed to be beneficial for both healthy and sick people alike. In short, meditation can supposedly make you mentally calmer, physically healthier, and cognitively better.</p>
<p>But can it really? If so, how? And which types?</p>
<p>Personally, I’ve been interested in meditation for awhile. A few years ago, <a href="http://www.markwk.com/2016/09/meditation-tracker.html">I meditated nearly daily</a> using some of the popular guided meditation and mindfulness apps available. Eventually, my interest and practice wained. I “felt” I had gained a few benefits, but I was a bit disappointed both by the time commitment and by the failure of my meditation practice to bring about more profound changes.</p>
<p>Perhaps I had the wrong expectations, perhaps I wasn’t meditating right, but increasingly I think my struggles around meditation reflect deeper questions about how to define what is meditation and all of the different types of meditations that exist. This difficulty to classify meditation is a sentiment echoed in the research on meditation too.</p>
<p>Infused in local culture and religion and bantered about in popular culture, meditation remains a pretty poorly defined term. In spite of the long tradition and a lot of current research, “there is no consensus on a definition of meditation in the scientific literature” (Ospina et al, 2008). Basically, <strong>no standard definition of meditation exists</strong>.</p>
<p>There are similar issue with the word “mindfulness” that has long left whatever it meant historically and philosophically and now seems to apply to just about everything including staying in the moment as you wash the dishes or eat a meal.</p>
<p>In all this confusion, I want to know: What is meditation? What are the types? And how do they affect us and our brains?</p>
<p>For our purposes and as I’ll explain in detail below, I define meditation as:</p>
<p><strong>A multi-step process whose two principal components are 1. the methods or cognitive strategies used and 2. the enhanced mental states it brings about.</strong></p>
<p>Defining meditation is a challenge. Basically, I’ve come to realize the importance of distinguishing between different techniques and methods of meditating and their intended goals. There might be benefits that go beyond meditating itself (a claim we will look at elsewhere), but at its origins meditation was intended as a mental practice that brought about a new kind of thinking or a new state of consciousness. Buddha and many others were meditating to reach an understanding of themselves and of the universe.</p>
<p>So, before we can examine the benefits, let’s try to define meditation first.</p>
<p>In the rest of this post, I’m going to walk-through how I came up with this definition and look at the two key parts: methods and enhanced mental states. After settling on a naive definition, we are also going to briefly survey several popular types of mediation and then look at some key features that are common to most meditative experiences. I’ll do my best to include various alternative definitions I found along the way.</p>
<p>Hopefully by the end of the post, we’ll be equipped with not only an operational definition of meditation but we should be able to answer the question why and for what purpose we are meditating for. Furthermore, I think this kind of definition is a good starting point for other technological, biological, or pharmaceutical methods in pursuit of cognitive enhancement or other phenomenological experiences.</p>
<!--more-->
<h3 id="a-naive-definition-of-meditation">A “Naive” Definition of Meditation</h3>
<blockquote>
<p>The word “meditation” is derived from the Latin “meditari,” which means “to engage in contemplation or reflection.” (Ospina et al, 2008)</p>
</blockquote>
<p>In the West a major growth of meditation began in the 1960s, which continues into today. Scientific research on meditation is believed to have started in the early 1930s but gained prominence in 1970s and 1980s and is still trying to understand the mechanisms behind meditation and its benefits.</p>
<p>First off, in a naive sense, what is meditation and what are we doing when we do it?</p>
<p><strong>Generically, meditation is a mental practice where a person trains their attention and awareness</strong>. Beyond that, definitions can go in various directions based on the belief system, technique or goal. Often times it involves sitting in one position in quiet but its goal might vary widely. Some might pursue it with focused attention, others with an open awareness, while another group might pursue a null, non-conceptual mental state.</p>
<p>Quite a few attempts have been made to define meditation in the scientific community (Cardoso, 2004; Bond, 2008; Ospina, 2008; Awasthi, 2013; Nash, 2013). One of the challenges is how to distinguish meditation from other stuff. This is referred to as demarkation and for many the best definition of meditation should allow us to operationally separate it from other types of mental and muscle relaxation (Bond, 2008). So, often the question could be put as, what distinguishes meditation from closed eyed relaxation?</p>
<p>In their meta-analysis, Ospina et al. (2008) call meditation an “umbrella term” and go on to offer a definition of meditation that focuses on shared properties across the different types:</p>
<blockquote>
<p>Despite the lack of consensus in the scientific literature on a definition of meditation, <strong>most investigators would agree that meditation implies a form of mental training that requires either stilling or emptying the mind, and that has as its goal a state of “detached observation” in which practitioners are aware of their environment, but do not become involved in thinking about it.</strong> All types of meditation practices seem to be based on the concept of self-observation of immediate psychic activity, training one’s level of awareness, and cultivating an attitude of acceptance of process rather than content. (Ospina 2008)</p>
</blockquote>
<p>They also note that breathing, training, mantas (or their lack) relaxation, spirituality or belief, and attention (and its object) are important aspects of meditation and how to distinguish one type from another.</p>
<p>In line with this idea of mental training and stilling the mind, several researchers propose that what defines meditation is what they call “logic relaxation” (Cardoso, 2004; Bond, 2008). While some might also remark that meditation typically includes physiological relaxation, this term “logic relaxation” is a bit of a neologism intended to covey</p>
<blockquote>
<ol>
<li>Not “to intend” to analyzing (not trying to explain) the possible psychophysical effects;</li>
<li>Not “to intend” to judging (good, bad, right, wrong) the possible psychophysical</li>
<li>Not “to intend” to creating any type of expectation (Cardoso, 2004)</li>
</ol>
</blockquote>
<p>Put more simply, meditation, according to this criteria, involves a mental state of non-judgement and non-analysis, where you disable your logic and reasoning thinking and are just there. At least that’s my understanding. Personally I don’t find this criteria particularly helpful.</p>
<p>In summary, in spite of the long tradition and current wealth of research, terms like “meditation” and “mindfulness” remain ill-defined. Colloquially, meditation can also beyond the practice of silently seated contemplation and often refers to things like relaxing or even analytically thinking about or explaining something in marked detail.</p>
<p>The plethora of meditation practices and lack of a standard definition of meditation <em>as such</em> creates challenges both for the practitioner and for the researcher, because even in everyday conversation, we can’t be sure if we are referring to the same thing when we say someone is meditating or that meditation is good for you.</p>
<p>Let’s start by taking a look at some of the most popular meditation types.</p>
<h3 id="popular-types-of-meditations">Popular Types of Meditations</h3>
<p>The practice of meditation is ancient, even prehistoric. The origins of meditation date at least to India. The first written evidence has been found in the Vedas in India (around 1500 BCE). While its primary site of development and home found in Asia, meditative practices have been a part of numerous cultures and traditions, including India, China and Japan. Early and Middle Ages Jewish, Sufi, and Christians all have both meditative and mystical traditions.</p>
<p><strong>So, before we try to define it better, just how many ways to meditate are there?</strong></p>
<p>This might seem like a trick question. One might argue that everyone meditates differently. Meditation is a unique and personal process. That might be true, but it doesn’t really help us if we want to understand meditation scientifically and if we ever hope to find a repeatable way to meditate with predictable health benefits and shared human experiences.</p>
<p>By my rough count, there are at least a dozen or so well-known meditation types people practice today. There are likely hundreds, if not thousands, more if we look at it from one of the active meditative traditions in Japanese, Chinese or Tibetan Buddhism, as a starting point. According to one major meta-analysis, 32 common meditation practices were identified from the scientific literature and from which, they emerged with 5 broad meditation types: Mantra Meditation, Mindfulness meditation, Qi Gong, Tai Chi and Yoga (Ospina et al, 2008).</p>
<p>In one study focused on Western Buddhist meditators and the variety of experiences, including negative ones, the researchers identified the following types of meditation practices along with the following the distribution among a group of meditators (Lindahl, 2017):</p>
<p><img src="/generated/images/2019-resources/meditation-types-800-38a4c7ccc.jpg" srcset="/generated/images/2019-resources/meditation-types-400-38a4c7ccc.jpg 400w, /generated/images/2019-resources/meditation-types-600-38a4c7ccc.jpg 600w, /generated/images/2019-resources/meditation-types-800-38a4c7ccc.jpg 800w, /generated/images/2019-resources/meditation-types-1000-38a4c7ccc.jpg 1000w" /></p>
<p>As you can see from this survey, Buddhist meditation includes many of the popular meditation types today. Though they have now been secularized format, mindfulness, body scan, insight, concentration and breath count are all Buddhist types too. For example, most of the mindfulness meditation today is based on samatha concentration meditation, a type of single-pointed mind practice commonly done through attentive mindful breathing.</p>
<p>While not meant to be comprehensive, here is my list of ten common types of meditation:</p>
<ol>
<li><strong>Concentration</strong> (Samatha, mindfulness of breathing, breath counting): Based on Buddhist version, this closed-eyed meditation focuses on training yourself to pay attention to your breathing. When you get distracted, return to the breath. This is called in the academic literature as focused attention meditation</li>
<li><strong>Insight</strong> (noting, open monitoring, Vipassana): In contrast to concentrative meditation, this is a non-directive meditation, where your attention is open and you strive to remain aware of everything that is happening.</li>
<li><strong>Body Scan</strong> (includes Goenka vispassana): This is a mediation where you focus on your body and scan through different parts of it through a prescribed narrative.</li>
<li><strong>Loving-kindness</strong> (Metta, compassion, karuna): A visualization meditation with goal of generating certain emotions.</li>
<li><strong>Zazen</strong> (Zen Buddhist tradition of Japanese lineage), including a few types but most notable is koan technique involving introspection on a paradoxical riddle.</li>
<li><strong>Mantra Recitation</strong> (“OM” Chanting or Transcendental): A typically closed-eyed meditation using a repeated a mantra with the goal of transcending the self.</li>
<li><strong>Visualization Practices</strong>: This is a bit of a grab-bag name for various meditation techniques often done with a guide.</li>
<li><strong>Walking Meditation</strong>: Like Tai Chi (Tai Qi Chuan) or Yoga, this meditation includes a kinetic or movement component.</li>
<li><strong>Qigong Breathing</strong>: A Taoist breathe-focused meditation aimed at harnessing energy (Qi) in the body through certain energy pathways called meridians.</li>
<li><strong>Nature of Mind Practices</strong> (dzogchen, mahamadra): These are meditation styles found in several traditions (Tibetan, Zen) and are focused on non-conceptual side. They strive to look at ones mind and go beyond duality of even existence and non-existence.</li>
</ol>
<p>Other notable types are Guided Meditation, Neurofeedback (Muse), Analytical Meditation (Tibetan Buddhist type that seems more like reflection than meditation), Sacred Silence, and Tonglen (Tibetan for ‘giving and taking’) among many others. Yoga and Tai Chi are also included in some literature as forms of meditation, though the physical component makes it problematic, since it’s difficult to separate the health benefits of the meditation part from the physical.</p>
<p>So, if we want to come up with a good working definition of meditation, it should at least be capable of include these major types.</p>
<h3 id="mindfulness-and-mindfulness-meditations-an-ambiguous-amalgam">Mindfulness and Mindfulness Meditations: An Ambiguous Amalgam</h3>
<p>As you might have noticed, I didn’t specifically include mindfulness meditation in the list above. In spite of the popularity of the term in general language and most meditation apps, the current usage of “mindfulness” is quite new, dating from sometimes after 1980s.</p>
<p>The term “mindfulness” is believed to be originally derived from “sati” (“smṛti” in Sanskrit), the Pali word in Indian Buddhism that has to do with a mental quality related to paying attention and staying present. It’s associated with staying curious, present and open.</p>
<p>While it is based on Zen, Vipassanā, and Tibetan meditation techniques from Asian cultures, mindfulness as it is now used is a Western fusion and creation. Several individuals have been heavily involved in its usage and growth. Notable figures include Thích Nhất Hạnh (a Vietnamese Buddhist monk), Jon Kabat-Zinn (creator of mindfulness-based stress reduction (MBSR), a popularly used medical application of mediation), and Richard J. Davidson (a professor, meditation researcher and writer of a recent book, “Altered Traits”).</p>
<p>As various researchers have noted, mindfulness is not a unitary construct or concept. It describes a few different mental features, like attention, presence and and being open and nonjudgmental. As one group put it:</p>
<blockquote>
<p>One of the most thoughtful and frequently invoked definitions states that <strong>mindfulness is moment-to-moment awareness, cultivated by paying attention in a specific way, in the present moment, as nonreactively, nonjudgmentally, and open- heartedly as possible</strong> (Kabat-Zinn, 1990, 2011). (Van Dam, 2018)</p>
</blockquote>
<p>Similarly, the practice of mindfulness meditation comprises Vipassana, Zen Buddhist meditation, Mindfulness-based Stress Reduction (MBSR), and Mindfulness-based Cognitive Therapy (MBCT). The actual practice of mindfulness mediation seems to combine several of the techniques we listed about, including concentrative, insight and others.</p>
<p>Put another way, mindfulness can potentially describe both or either a mental construct or the practices you do. It’s a curious and ambiguous amalgam of both the mental state aimed at (i.e. being in a mindful state) and the practices you pursue (“being mindful” or “minding”).</p>
<p>If this seem slightly confusing, you are not alone. Many researchers rightly argue that, in spite of the popularity of mindfulness in society and scientific literature, it’s a term plagued with semantic and scientific misunderstandings. The confusion around this word and descriptions of meditation practices themselves are one of many concerns among researchers about the wild claims about the benefits of meditation too.</p>
<p>As such, many researchers believe mindfulness is a rather poor or at least limited way to describe meditation and the target mental states. As such, many believe mindfulness shouldn’t be use and instead we should strive to define meditation in a more robust way, namely based on either the features involved (Van Dam, 2018) or as a multiple-step process (Nash, 2013).</p>
<h3 id="a-scientific-list-of-key-features-in-meditation">A Scientific List of Key Features in Meditation</h3>
<!--[[201906271110_meditation_features]]-->
<p>The health and cognitive benefits of meditation appears to be one of most widely held ideas today. But in one of the largest meta-analyses ever done on meditation and its health benefits (Ospina et al, 2008), researchers reviewed over 800 studies and noted three principal concerns: First, they found “the methodological quality of meditation research to be poor, with significant threats to validity in every major category of quality measured, regardless of study design.” Second, while the most consistent physiological effects were “reduction of heart rate, blood pressure, and cholesterol” and the “strongest neuropsychological effect…in the increase of verbal creativity,” they concluded:</p>
<blockquote>
<p><strong>The field of research on meditation practices and their therapeutic applications is beset with uncertainty. The therapeutic effects of meditation practices cannot be established based on the current literature.</strong> (Ospina, 2008)</p>
</blockquote>
<p>Third and finally, they remark that more research is needed which must include a “clear conceptual definition of meditation” itself.</p>
<p>These conclusions and the underlying sentiment was confirmed in a recent and aptly titled paper, “Mind the hype,” in which a group of researchers argued not just for skepticism about the benefits and research methodology but the definitions used around meditation too (Van Dam, 2018). Fortunately, the researchers did not just end with a critique but also attempted to classify and define the key features involved in all types of meditation.</p>
<p>Van Dam and others focused their classification on the primary features of meditation practices themselves and those involved (participants and teachers) as well as the secondary characteristics of meditation/mindfulness interventions. This allowed them to come up with the following list:</p>
<h5 id="primary-features">Primary features:</h5>
<ul>
<li>Arousal: how alert/awake (low, medium, high)</li>
<li>Orientation (of attention): where is attention directed (inward vs. outward vs. no orientation)</li>
<li>Spatial dynamic (of attention): Is it fixed on a single object (like breath or a mantra) or dynamic (like a body scan or visualization practice)?</li>
<li>Temporal dynamic (of attention)</li>
<li>Object (of attention)</li>
<li>Aperture (of attention): Is one’s attention focus narrow, intermediate or diffuse?</li>
<li>Effort: How hard are you trying to exert energy to achieve other features?</li>
</ul>
<h5 id="secondary-features">Secondary features:</h5>
<ul>
<li>Complimentary activity (like walking, mantra or recitation)</li>
<li>Affective valence: Is your practice positive, neutral or negative?</li>
<li>Emotional intention: Is the intention loving-kindness, generosity, etc?</li>
<li>Motivation / goal: What reason goes into the practice? Self-improvement, enlightenment, wellness, etc.?</li>
<li>Posture (lying down, sitting, standing)</li>
</ul>
<p>The researchers go on to list various other criteria, like the number of participants and teacher. If this list seems exhaustive, then it’s important to note that the intention is to help guide better researchers in their classifications and defining of the meditation practices being studied. Specifically, they want to provide a way that any meditation benefits that are claimed in a study can be repeated and restudied, a central tenant in scientific method.</p>
<p>Unfortunately, while this list can be helpful in some ways, especially for researchers, I find it a bit limiting personally in our pursuit of what meditation is. It also fails to account for a point brought up by Ospina et al. (2008), namely what are the “criteria of successful meditation practice”? As they put it:</p>
<blockquote>
<p>The criteria of successful meditation practice are understood both in terms of the successful practice of a specific technique (i.e., is the technique being practiced properly) and in terms of achieving the aim of the meditation practice (e.g., has practice led to reduced stress, calmness of mind, or spiritual enlightenment). (Ospina, 2008)</p>
</blockquote>
<p>Both personally and intellectually I find this question about “successful” meditation to be a challenge one. I often hear that meditation should be about enjoying the process and staying in the moment, which might be true and beneficial, but I find the lack of criteria that defines the quality and success of meditation to be problematic.</p>
<p>Personally, how can I know if my meditation practice went well in a single session? How can avoid wasting hours meditating in a way that doesn’t work?</p>
<p>Furthermore, if we want meditation have a positive health and societal impact, how to determine whether the meditation practice was done right or wrong and whether it worked or didn’t?</p>
<p>With these questions in mind, let’s turn to a definition of meditation as a multi-step process.</p>
<h3 id="the-science-of-ancient-methods-in-pursuit-of-enhanced-mental-states-defining-meditation">The Science of Ancient Methods In Pursuit of Enhanced Mental States: Defining Meditation</h3>
<p>Academic papers on meditation often define meditation from either the perspective of the subject (meaning the cognitive state they are in their mind) or in terms of the methods used (chanting, breath counting, etc.). Both have their merits. For example, by listing the features in detail you can better define what is happening. Similarly, by pointing to a meditative state, you might be able to causally argue that benefits come from reach and staying in a certain mental state.</p>
<p><img src="/generated/images/2019-resources/stages-of-meditation-process-456-af8e4a2b7.jpeg" srcset="/generated/images/2019-resources/stages-of-meditation-process-400-af8e4a2b7.jpeg 400w, /generated/images/2019-resources/stages-of-meditation-process-456-af8e4a2b7.jpeg 456w" /></p>
<p>In “Toward a unifying taxonomy and definition for meditation” published in 2013 in Frontiers in Psychology, Nash and Newberg provide one of the best formulations I’ve read on what is meditation and how to define different practices using a taxonomy. They argue that meditation should not be defined by either a meditative state nor a particular method, but instead a model of meditation should encompass both. For them, <strong>meditation should be conceptualized as “a dynamic process which is inclusive of six related but distinct stages”</strong> (Nash, 2013).</p>
<p>Leaving aside the full six stages in their workflow, two key components in their taxonomy stand out: enhanced mental state (EMS) and the method.</p>
<h5 id="meditative-methods-our-cognitive-attention-strategies">Meditative Methods: Our Cognitive, Attention Strategies</h5>
<p>When we think about meditation we often focus on the how-to. We want to focus on what we are supposed to do. Some examples might be being mindful and attentive to our breathing, body scan, breath counting or chanting. These are all different techniques and methods.</p>
<p>Methods generically stand for the steps we do to accomplish something. When applied to meditation, <strong>methods are the “a family of complex emotional and attentional regulatory training regimes developed for various ends, including the cultivation of well-being and emotional balance”</strong> (Lutz et al., 2008b, p. 163, quoted in Nash, 2013).</p>
<p>Simply put, <strong>methods are the mental practices we undertake and cognitive strategies we deploy while meditating</strong>. From that point on, the researchers argue that meditation practices stipulate a causal link between the method and the outcome of that method.</p>
<h5 id="meditative-mental-states-the-cognitive-mental-state-of-our-practice">Meditative Mental States: The Cognitive Mental State of Our Practice</h5>
<p>Each type of meditation has a goal. While often forgotten and rarely mentioned in many mindfulness apps and guided meditations tools today, each meditative practice intends to bring you to a certain meditative state, altered state of consciousness, or mental state. This mental state should be viewed as separate from the potential health and cognitive benefits of meditation.</p>
<p>Each method and meditation type is different, but for every single one, the successful application of its meditation method should help you reach an intended mental state that Nash and Newberg call the <strong>enhanced mental state (EMS)</strong>.</p>
<p>Returning briefly to the stages, Nash and Newberg provide the following working definition of meditation:</p>
<blockquote>
<p><strong>A given meditation session is defined as the time allocated by the meditator to the engagement of the process; whereby the meditator starts from a mundane state of alert/waking consciousness, moves through the specific stages of the process over time, and then returns to that same state of waking consciousness.</strong> (Nash, 2013)</p>
</blockquote>
<p>As we all know, meditation starts with your intention to meditate. You then make certain preparations and rituals (seated position, turn on app, candles, music, etc.).</p>
<p>Once you start meditating, you deploy attention and self-regulatory mental activities and strategies for that type of meditation. These cognitive strategies are provided by a teacher or didactic material, like a book, audio file or even app. A few that Nash and Newberg mention include concentration, mindfulness, open-monitoring, breathing, chanting and many others. These are the methods we refer as mental training but really they are just subjective ways to monitor and manage our mental happenings.</p>
<p>Even though there is a lot of interest today around the health and medical benefits of meditation, especially for stress, we mustn’t forget that original and historical intent of most meditation was a spiritual one. For example, the Buddha and others were seeking spiritual awakening and enlightenment.</p>
<p>In this same spirit it is important to remember that each meditative practice has an intended mental state it seeks to achieve using its method. For Nash and Newberg, <strong>the enhanced mental state (EMS) is the meditative state where the “subjective first-person reports of a shift in consciousness to a different and more “profound” state such as: an enhanced sense of well-being, focus, calm, detachment, insight, affect, bliss, emptiness, etc.”</strong></p>
<p>Essentially, while it might be fleeting and even non-existent in a new practitioner or sustained for an advanced/experience meditator, the purpose of meditating is to bring you into a different, non-normal state of mind or consciousness.</p>
<p>In their attempt to classify meditation types, the researchers explicitly mentioned three broad states:</p>
<ol>
<li><strong>Enhanced affective state (EAS)</strong>: In this type of meditation your goal is reach a state of compassionate awareness tied to you and your social world. Metta, compassion and loving-kindness meditations from Tibetan and Theravada Buddhism would be a few good examples.</li>
<li><strong>Enhanced cognitive state (ECS)</strong>: A type of meditation that purports to engender mindfulness and insight, this is arguably the most common type of meditation methods practiced by most people in the West today. The religion tradition in samatha and vipassana are two examples of this a general purpose mindfulness meditation in which you cognitively pushed you towards a deeper religious or spiritual point of understanding.</li>
<li><strong>Enhanced non-cognitive/Non-affective state (NC/NA)</strong>: This type of meditation strives to bring you into an enhanced “empty” or “null” state, one that is devoid contents. This is the no-mind that some mystics might call God Consciousness among other terms. Poorly translated in most Western languages, this includes words like nirodha-sama ̄patti (Pali), samadhi (Sanskrit), satori (Japanese), dzogchen (Tibetan). It is meant to convey an ineffable and non-conceptual state of consciousness.</li>
</ol>
<p>The rest of the paper, which I highly recommend for anyone wishing to get an even more technical survey, goes into the neurological components. By framing meditation as a multiple-step process, it’s possible to point to neurological correlates for each of the meditative stages and different mental states too. This quest for the neurology of meditation is critical if we hope to fully understand meditation and the human mind as such (Awasthi, 2013).</p>
<p>One unfortunate thing missing in the Nash and Newberg paper is a final general definition of meditation. Admittedly, they do provide detailed classifications of some of the popular types of meditation and organize them according to their methods and their intended mental states, which is quite useful.</p>
<p>Fortunately, based on what I hope is a faithful reading of their research, we can venture a pretty solid definition of meditation now ourselves:</p>
<p><strong>Meditation is a multi-step process whose two principal components are the methods or cognitive strategies used and the enhanced mental states it brings about.</strong></p>
<p>By formulating meditation in this way, we can operationally ask and answer two things about different meditations types we might try: the how and the why.</p>
<p>First, on the question of how to meditate, this depends on the particular type but we now can point to the methods and techniques used in different meditation types. Open-monitoring or focused attention would be two options in a long array of cognitive strategies used.</p>
<p>Second, on the question of why meditate (or, as I put it, what am I meditating for?), by our definition, we are meditating to reach a certain meditative state or, as the researchers put it, an Enhanced Mental State. They mention mental states like a calm, relaxed state of open awareness, a focused attention on your breath and body, or a null state of non-conceptuality, but they also indicate the possibility of many others.</p>
<p>Additionally, we might claim we meditate for the benefits, which interestingly might be due to either the method or the mental state. Researchers sometimes refer to these as altered states and altered traits. Altered states have been what we commonly think about as the purpose of meditation (as well as other psychotropic methods). We meditate for a mental state. Increasingly, many researchers seem to believe meditation brings with it “altered traits,” which is also the title of a recent book on meditation by well-known researchers. For them, altered traits are the health, emotional and cognitive changes people can get from meditating. These are bold claims that we will need to explore elsewhere.</p>
<h3 id="conclusion-meditation-as-methods-towards-cognitive-enhancement">Conclusion: Meditation as Methods Towards Cognitive Enhancement</h3>
<p>I’ll admit to a long-time fascination with the human mind and how to improve it. I’ve applied this to various endeavors, like learning languages, writing, and creativity. In <a href="http://www.markwk.com/science-of-learning.html">Learning How to Learn</a> I explored how our brains learn and noted a few areas that can help us learn better, like exercise, taking breaks and sleeping. This lead to my own efforts to sleep and exercise more, since it improved my mind!</p>
<p>Understanding and improving my mind is one of the reasons I started meditating in the first place. But after meditating regularly for a few years, I stopped. Had I improved my mind? Had meditation brought me the benefits I had hoped?</p>
<p>Honestly, I’m still not entirely sure. Cognitive enhancement is a term often used to describe the conscious attempt to improve one’s mind, thinking, creativity, and more. There are several lifestyle approaches that can improve your learning and mind, like sleep, exercise, a conductive environment, certain habits, and even music. Many transhumanists and biohackers use nootropics or “smart drugs” to pursue mental acuity too.</p>
<p>Besides the reported health benefits, meditation seems to offer an interesting route towards cognitive enhancement too. This might include both the benefits of leveraging certain mental states during actual meditation, but it might also have neuropsychological advantages too that transfer from meditation itself to our non-meditation activities.</p>
<p>Does meditation improve our health? Does it improve our thinking and cognitive functions? Does the improvements in our meditation practice itself transfer to our work and learning?</p>
<p>Now that we have a definition of what is meditation, we can start to think and examine these questions in more detail. Many of these question I hope to follow up on more soon, especially since I started meditating again and am now using a brainwave monitor to help me quantify my practice.</p>
<p>In this post, we ventured through a series of questions on what is meditation. Starting with a pretty naive definition of meditation, we looked at some popular types and typical features, before reaching our final working definition of meditation as a multi-step process primarily involving different mental practice (methods) that are intended to engender certain cognitive states (enhanced mental states).</p>
<p>In short, you might define meditation as methods used towards cognitive enhancement, a definition might make it attractive to biohacker and data-driven self-improvers too. Obviously this definition leaves some parts out, but I think it works well for my purposes of thinking about meditation in terms of methods, mental states and benefits.</p>
<p>Much of the research I’ve looked at on meditation tends to be at either extreme. Some argue for a plethora of benefits and are able to find some degree of science to support it. Similarly, as we looked at here, quite few researchers, including some substantial reviews, have found the research and health claims on meditation to be suspect. There is also a few research efforts show casing some of the negative side effects and experiences in meditation too (Lindahl, 2017; Kornfield, 1979).</p>
<p>While we will need to look at health and cognitive benefits of meditation separately, Ospina et al. (2008) in their detailed review of 800 meditation studies offer the following summary:</p>
<blockquote>
<p><strong>Our meta-analysis revealed that the most consistent and strongest physiological effects of meditation practices in healthy populations occur in the reduction of heart rate, blood pressure, and LDL-C. The strongest neuropsychological effect is in the increase of verbal creativity.</strong> There is also some evidence from before-and-after studies to support the hypothesis that certain meditation techniques decrease visual reaction time, intraocular pressure, and increase breath holding time. Though over half of the combined effect estimates are not statistically significant, the potential clinical significance of these estimates must be carefully considered. <strong>However, all of the studies included in the meta-analyses were of low methodological quality and, for this reason, the results should be interpreted cautiously.</strong> (Ospina, 2008)</p>
</blockquote>
<p>I’d say the evidence is still out on the benefits of meditation as such. We do not yet know if or how meditation changes us for better (or worse). But in thinking about these questions, I’ve come to realize that it’s not just about the benefits themselves. We might hope there are benefits and that they are transferrable outside of our practice, but we shouldn’t forget that part of meditation’s importance is in the experiences and mental states it brings. In fact, meditation is one of the longest on-going human traditions and practices in existence whose purpose was and is to bring us into certain mental states.</p>
<p>Personally, one thing I have learned in researching meditation and trying to define it is that there are a lot more techniques to meditation than just setting quietly and trying to be mindful. That is definitely one way but not the only way to meditate.</p>
<p>Moreover, if you look at meditation holistically in terms of methods and induced mental states, you realize there are potentially endless way to train, practice, induce or guide a mind. I definitely plan to explore and try some of this other meditative methods.</p>
<p>Finally, when I look again at our definition (a multiple step process of methods that brings about new mental states and experiences) and think about it more broadly, I realize that it applies beyond just meditation. It applies to any practice that changes us. In fact, it can apply to a whole host of approaches: biological, <a href="http://www.markwk.com/multistage-goal-pursuits.html">goal-driven</a>, experiential, pharmaceutical, <a href="http://www.markwk.com/quantified-self-mind-map.html">data-driven</a>, or even technological.</p>
<p>In my opinion, we should embrace this and pursue the wealth of human experiences possible both in meditation and in life in general.</p>
<p>Best of luck and happy meditating!</p>
<p><br /></p>
<p><br /></p>
<h5 id="references">References:</h5>
<ul>
<li>Awasthi, B. (2013). Issues and perspectives in meditation research: in search for a definition. <em>Frontiers in psychology</em>, <em>3</em>, 613.</li>
<li>Bond, K., Ospina, M. B., Hooton, N., Bialy, L., Dryden, D. M., Buscemi, N. et al. (2009). Defining a complex intervention: The development of demarcation criteria for “meditation”. <em>Psychology of Religion and Spirituality</em>, <em>1</em>(2), 129.</li>
<li>Cardoso, R., Souza, E. D., Camano, L., & Leite, J. R. (2004). Meditation in health: an operational definition. <em>Brain research protocols</em>.</li>
<li>Chiesa, A., & Serretti, A. (2014). Are mindfulness-based interventions effective for substance use disorders? A systematic review of the evidence. Substance use \& misuse, 49(5), 492-512.</li>
<li>Goleman, D., & Davidson, R. J. (2017). <em>Altered Traits: Science Reveals How Meditation Changes Your Brain, Mind, and Body</em>. Penguin.</li>
<li>Goyal, M., Singh, S., Sibinga, E. M. S., Gould, N. F., Rowland-Seymour, A., Sharma, R. et al. (2014). Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA internal medicine, 174(3), 357-368.</li>
<li>Kornfield, J. (1979). Intensive insight meditation: A phenomenological study. The Journal of Transpersonal Psychology, 11(1), 41.</li>
<li>Lindahl, J. R., Fisher, N. E., Cooper, D. J., Rosen, R. K., & Britton, W. B. (2017). The varieties of contemplative experience: A mixed-methods study of meditation-related challenges in Western Buddhists. PloS one, 12(5).</li>
<li>Nash, J. D., & Newberg, A. (2013). Toward a unifying taxonomy and definition for meditation. <em>Frontiers in psychology</em>, <em>4</em>, 806.</li>
<li>Ospina, M. B., Bond, K., Karkhaneh, M., Tjosvold, L., Vandermeer, B., Liang, Y. et al. (2008). <em>Meditation Practices for Health: State of the Research</em>. Evidence Reports/Technology Assessments, No. 155. Agency for Healthcare Research and Quality (US). Available Online a https://www.ncbi.nlm.nih.gov/books/n/erta155/.</li>
<li>Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D., Olendzki, A. et al. (2018). Mind the hype: A critical evaluation and prescriptive agenda for research on mindfulness and meditation. Perspectives on Psychological Science, 13(1), 36-61.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Meditation is one of the most touted habits we are all supposed to do these days. Meditation and mindfulness are lauded for a range of associated benefits from physical and mental health to cognitive improvements and beyond. It’s believed to be beneficial for both healthy and sick people alike. In short, meditation can supposedly make you mentally calmer, physically healthier, and cognitively better.Data-Driven Health: Are you tracking a health indicator or healthy habit?2019-06-28T00:00:00-07:002019-06-28T00:00:00-07:00http://www.markwk.com/tracking-health-or-healthy-habits<p>There is a lot of excitement around <em>how</em> technology can be applied to improve our health. It seems like every day there is some new technology, wearable, pill, test, service or app aiming to track, monitoring, improve or cure us of each and every bodily aligment. There is even a growing number of companies and tools aiming at human optimization, neuroenhancement, and peak performance, sometimes called “biohacking.”</p>
<p>For basically all of health tech, the goal is to improve our health, and, whether its a wearable, test or something else, each is claim to be tracking some our health.</p>
<p><strong>But what are these technologies tracking? And how do they fit into the pursuit of data-driven health and wellness?</strong></p>
<!--more-->
<h3 id="biohacking-and-endless-health-trackers-but-where-has-it-gotten-us">Biohacking and endless health trackers, but where has it gotten us?</h3>
<p>The focus in the healthcare and health tech space has largely been on both major diseases and the pursuit of healthier, active lives. Hospitals, doctors and other health care professionals largely focus on treating diseases, while trainers, coaches, nutritionists, teachers and some advocates of functional medicine dedict their work to fitness and wellness. Increasingly though we see companies, blogs and services dedicated to what is often called “biohacking.”</p>
<p>Biohacking refers to the idea of applying IT and tech “hacks” to biological systems, most notably our own bodies. Basically, biohacking is the practice of changing your biology and environment towards the express aim of controlling and enhancing your body, mind and life. While I rarely use that word, biohacking aligns with various aspects of what I call self-tracking, the quantified self, and “data-driven life,” especially in how it relates to our health.</p>
<p>But where has all of these health-related tools, wearables and services gotten us?</p>
<p>Obesity continues to rise. Acording to the W.H.O (2018), the world has seen a 10x increase in number of obese children and adolescents ages 5-19 in the past four decades, going from just 11 million in 1975 to 124 million in 2016. All told, that equates to 1 in 5 of children globally are overweight or obese. According a 2009 report from the CDC, 65.7% of American adults were estimated as overweight in 2010, which is a marked increasing trend from 39.4% in 1997, 44.5% in 2004, and 56.6% in 2007.</p>
<p>Basic good nutrition still manages to baffle most “normal” folks. Anyone who follows nutrition to some extent in the media could feel overwhelmed by seeming flip-flops on what sciences says about eating do’s and don’t’s. At least once I year I have to google some new diet or eating protocal I hear about from someone. In spite of a growing consensus about the basics of good nutrition (see <a href="https://www.theatlantic.com/health/archive/2017/09/moderate-intake-of-things-linked-to-health/538428/">this article in the Atlantic</a> or a <a href="http://www.grubstreet.com/2018/03/ultimate-conversation-on-healthy-eating-and-nutrition.html">well-written dialogue</a> for no-non-sense summaries), we people’s eating habits are pretty lousy.</p>
<p>What’s to be done?</p>
<h3 id="wearables-to-the-rescue-maybe-not-so-fast">Wearables to the rescue! Maybe not so fast…</h3>
<p>One of the most commonly reported “heath” trackers used today is wearables. Nearly 1 in 4 American adults have and use a wearable with even higher rates in Asia with over 50% in China and India among others, according a recent report from eMarketer (2019 ). This wearable usage trend continues to rise too.</p>
<p>In fact, a survey of 16 Western countries that went beyond just wearables found that :</p>
<blockquote>
<p>one in three people track their health and well-being with the help of a Quantified Self tool (i.e., online or mobile applications, activity trackers, smartwatches, or clips), and activity trackers represent more than 50% of the wearable market. (quoted in Stiglbauer, 2019)</p>
</blockquote>
<p>In spite of the seemingly endless options for tracking, wearables, and “biohacking” your health, a good deal of skepticism remains. And rightly so. When it comes to health tech and especially wearables, we’ve already gone through a couple hype cycles over the years. Everyone signals out the need to show themselves wearing this or that wearable, but then complain it annoys them with notifications or they are can’t understand why their weight hasn’t changed.</p>
<p>Even the medical results have been somewhat lackluster when it comes to health benefits of a wearable and health tracking apps (see Stiglbauer, 2019). Nearly everyone who is slightly in the technology space already has at least one forlorn wearable laying about or some installed health app that sits on your homescreen unused.</p>
<h3 id="wearables-dont-make-you-healthier">Wearables don’t make you healthier.</h3>
<p>Frankly, while a few studies have shown promise and a better way to conceptualize their use (Hermsen, 2017), it shouldn’t take an academic meta-analysis to realize wearables aren’t a magic bullet to better health.</p>
<p>First, merely having a activity tracker doesn’t change the underlying habits we must undertake to be active, get exercise, sleep, and eat right. Having a thing on your wrist can’t make you do this (at least not yet).</p>
<p>Second, wearables don’t track your health status anyways. In fact, in most dimensions, wearables and much of consumer medtech weren’t designed to monitor health status or provide medical-grade health indicators.</p>
<p>Excluding ECG or pulse oxyimeter in some devices, these device are actually tracking your commitment and enactment of healthy habits. Admittedly your movement and sleep history might be used to see trends and interpret your heath status or underlying factors contributing to your health situation, but so far we’ve yet to see a publicly sold wearable that predicts either how long you will live or a medically veriable standard of your health situation. That’s just not what they do.</p>
<p>So, this leads me to one question we all should be asking: <strong>What are we tracking when we track and monitor our health?</strong></p>
<h3 id="tracking-a-health-status-or-a-healthy-habit">Tracking a Health Status or a Healthy Habit?</h3>
<p>Health is the most motivation for <a href="http://www.markwk.com/why-people-self-track.html">why people self-track</a> and, along with fitness, it represents a huge portion of the wearables and <a href="http://www.markwk.com/quantified-self-mind-map.html">quantified self</a> space. We all want to be healthier, fitter and feel better. But not all health tracking services should be conflated.</p>
<p>In fact, there are many dimensions to track and monitor your health, and there are hundreds of companies and tools geared towards providing goods and services in all of those ways. But it’s time that we realized that not all health tracking is equal, nor should they be.</p>
<p><strong>We need to separate data-driven health into two groups: those that track and monitor our health status and those that track and support us in our healthy habits and commitments.</strong></p>
<p>By health status, I mean any number of assessments, tests and tools that allow you to diagnosis disease, determine how healthy/unhealthy/fit you are. You might instead call them health indicators or even better use my favorite term for them, <a href="http://www.markwk.com/what-are-biomarkers.html">biomakers</a>. Typically, you get your health status checked when you visit your doctor, get a blood test or measure your blood pressure.</p>
<p>Health status indicators are scientifically-validated standards of our health. First, they are backed by an accurate way to measure a part of your body and health giving you a number on a scale or range. Second, based on research studies, large populations and aggregate data, you and/or your doctor should be able to read and interpret where you stand health-wise.</p>
<p>For example, <a href="http://www.markwk.com/blood-pressure-tracking.html">blood pressure</a> is a well-established way to track and measure your health status. A normal resting blood pressure for adults is 120 systolic over 80 diastolic. 120/80 is healthy while higher than 120/80 is a potential sign of Prehypertension and a systolic between 140 and 159 is diagnosable as hypertension. In short these are actual numbers that indicate your situaiton. This is also possible for nearly all <a href="http://www.markwk.com/blood-test-faq.html">blood biomarkers</a> too.</p>
<p>Basically a health status indictator or a biomarker takes an accurate measurement that you can put in context to actually know if you are healthy or not. Similarly these numbers can be used across time to see trends and watch for changes.</p>
<p>By contrast, a lot of technologies and tools exist to support and track what we might refer to as our healthy habits and commitments. They include most wearables, smart watches, running and other fitness apps, meditation apps and whole range of things. These heathy habit trackers are great, and they save use mentral space by keeping a record of the activities we do to stay healthy, active and improve our fitness. They can also nudge you towards a better lifestyle (and, if you are like me, give you pretty charts and graphs too!).</p>
<p>But if we are honest, the vast majority of what we think of as “health trackers” don’t track how healthy or sick we are; they actually track our adherence to certain healthy habits, like daily steps, walking, or nighly sleep. We can know about a range of improvements in our fitness, even some that might be tied to our baseline health status (more on this shortly). But they can’t and do not indicate our actual health status.</p>
<p>So, if we return to the question of what we are tracking, most apps and wearables track our healthy habits, goals and committments. By contrast, tracking our actual health requires us to measure a biological indicator or biomarker that is indicative and predictive of our biological health situation. Most of these time means going to a doctor or clinic for a test, though greater access is arriving for home health care.</p>
<p>While this conceptual division between health status trackers and healthy habit trackers can help with understanding what and why you track certain things, this division isn’t always so black and white. Quite a few trackers could be used to either purpose. And some areas, like Sleep, Weight, Active Heart Rate, and Heart Rate Variability (HRV), are both logs of our healthy habits and commitments AND potential markers of our health status too. This points takes some explaining so let’s try and break it down more simply.</p>
<h4 id="how-tracking-a-healthy-habit-might-provide-a-health-status-indicator">How Tracking a Healthy Habit Might Provide a Health Status Indicator</h4>
<p>Most health tests monitor an aspect of our body, and most healthy habit trackers passively record our activity, steps, sleep, heart rate, and more. The reality is that most lifestyle diseases, like obesity, diabeties, heart disease and many others, which we can find indicators in medical tests, are the product of daily habits. For example, if you do little or no physical activity and eat poor (all unhealthy habits), then there is high statistical liklihood that your health status will be poor.</p>
<p>So, an interesting thing happens when we look at our regular activities tracked using a wearable, because they can often provide signals of worsening or improving healthy. On the negative side, chronic behaviors, sleep deprivation, high stress and inflammation, or not taking your medicine regularly, can can lead to medical conditions. At the same time, on the positive side, getting more activity and regular sleep might indicate an underlying health status improvement.</p>
<p>To put it in practical terms, excuding specific cases of major sleep disorders, we should group <a href="http://www.markwk.com/sleep-tracking-tools.html">sleep tracking</a> as a form of healthy habit tracking. The goal is to pursue the habit of sleeping more and we track to see how we are doing. But as multiple studies have shown not getting enough sleep and cronic sleep deprivate can lead to a range of health outcomes, like higher instances of cancer, higher probability of infection and colds and even worse mortality rate (Walker, 2017). This is one reason why I think, in spite of not enough discussion and user-friendly tools on how to do this, there is a lot of potential in applying sleep tracking to monitoring your health. This could similarly be applied to the sedentary life.</p>
<p>Similarly, most of fitness tracking is largely about tracking your habit, improvements and training. For example, how far or fast did I run or cycle? How many reps at a certain weight did I lift? For example, we might use running numbers for <a href="http://www.markwk.com/data-driven-run-training.html">data-driven marathon training</a>. Interesting there is one aspect or number in fitness tracking that can be used as a health indicator and biomarker, and that’s VO2 Max. <a href="http://www.markwk.com/2017/07/finding-my-vo2-max.html">VO2 Max</a> is your maximum-oxygen-processing capacity. It’s how well you can move oxygen while under stress. It varies from athlete to athlete depending on the level of cardiovascular fitness. It can be improved through training too. While at its core it represents your current fitness level, VO2 Max has also been studies by medical researchers and found to have predictive value about your health status, chance of certain diseases, heart health and even mortality risk.</p>
<p>When you look a list of health trackers and tools, quite a few could fit either or both designations. Weight could be used to classify obesity or track a fitness goal. <a href="http://www.markwk.com/science-of-hrv.html">Heart Rate Varability (HRV)</a> can also serve as a look at daily and chronic stress and other physical changes, like training adaptation. Interestingly even a medication or supplements reminder app, which largely serves the role of supporting a healthy habit of taking your vitamins and medicine, also has the potential tell you about your health status in view of how critical medication adherence is with certain diseases and treatments.</p>
<h3 id="data-driven-health-means-knowing-what-you-are-tracking">Data-Driven Health Means Knowing What You Are Tracking</h3>
<p>At the end of the day, personal data-driven health means knowing what you are tracking and towards what purposes. Is it helping you get an idea of how heathy you are? Or is it tracking your adherence to certain habits and health goals?</p>
<p>Overall, I think there is a lot of potential and promise in both sides of heath tracking. We are seeing more, cheaper, and better access to ways to monitor your health status as well as interesting medical usages of that data for a deeper look at health, wellness and disease.</p>
<p>Personally, even though I’m partialy to more advanced daily tracking methods and exploring new areas, I think the single best way to track our health status is still regular blood tests. Forget Theranos. Forget the multivitiamins. <a href="http://www.markwk.com/blood-test-faq.html">Get a regular blood test</a> and ask for a copy of the result. Then make lifestyle changes or take supplements based on your biomarker data, not because you read some article that says you should.</p>
<p>When it comes to personal trackers, wearables and other ways to measure my body and track life activities like <a href="http://www.markwk.com/how-to-track-workouts-fitness.html">workouts</a>, I’ll admit to an on-going obsession with the so-called tracking tech. But, hopefully as this post indicated I’m skeptical too.</p>
<p>To reiterate, in nearly all cases and devices, we aren’t tracking our health status when we wear an activity tracker or use some fitness app. Instead, these are tools that enable us to track healthy habits and fitness commitments. They collect data while we do that activity, like <a href="http://www.markwk.com/running-workout-types-for-training.html">running</a>, and keep a log. Some help us with a data-driven training plan. Mostly they can and should nudge us to be active and support us in positive habits and routines. For example, I appreciate how my watch reminds me to stand up regularly and encourages me to move about. Having a daily score works for me too. Similarly, my suppment tracker and <a href="http://www.markwk.com/2016/10/habit-tracking-for-quantified-self.html">habit trackers</a> help me when I am building a new habit and remind me when I forgot.</p>
<p>In fact, I’d argue that the principal benefit and focus of most health tech tools should be about <a href="http://www.markwk.com/science-of-goals.html">setting goals</a>, building habits and <a href="http://www.markwk.com/multistage-goal-pursuits.html">behaviorial change</a>. Getting people to create and stick to good habits and routines is the principal challenge of healthcare today. These are <em>really</em> hard challenges, but they are key if we want to pursue a meaningful data-driven health individually and , society-wide.</p>
<!--[[201906261239_data-driven_definition]] -->
<p>So next time you are thinking about your health and how to track or monitor it, be sure to ask yourself: <strong>Am I tracking a health indicator or healthy habit?</strong></p>
<p><br /></p>
<p><br /></p>
<p>Best of luck and happy tracking!</p>
<p><br /></p>
<p><br /></p>
<h5 id="academic-references">Academic References:</h5>
<ul>
<li>Ajana, B. (2017). <em>Self-Tracking: Empirical and Philosophical Investigations</em> (1st ed. 2018 ed.). Palgrave Macmillan.</li>
<li>Center for Disease Control (CDC). (2019). “U.S. Obesity trends”. Available Online at https://www.cdc.gov/obesity/data/index.html.</li>
<li>Cleland, I., Nugent, C., & Lee, S. (2016). The ground truth is out there: challenges with using pervasive technologies for behavior change. <em>Proceedings of the 10th EAI International …</em>.</li>
<li>Hermsen, S., Moons, J., & Kerkhof…, P. (2017). Determinants for sustained use of an activity tracker: observational study. <em>JMIR mHealth and …</em>.</li>
<li>eMarketer. (2019). Wearables 2019. Retrieved 6/28/19, https://www.emarketer.com/content/wearables-2019.</li>
<li>Stiglbauer, B., Weber, S., & Batinic, B. (2019). Does your health really benefit from using a self-tracking device? Evidence from a longitudinal randomized control trial. Computers in Human Behavior.</li>
<li>Walker, M. (2017). <em>Why We Sleep</em>. Simon and Schuster.</li>
<li>World Health Organization (WHO). (2018). World health statistics 2018: monitoring health for the SDGs, sustainable development goals.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}There is a lot of excitement around how technology can be applied to improve our health. It seems like every day there is some new technology, wearable, pill, test, service or app aiming to track, monitoring, improve or cure us of each and every bodily aligment. There is even a growing number of companies and tools aiming at human optimization, neuroenhancement, and peak performance, sometimes called “biohacking.”Quantified Self and Self-Tracking Mind Map: Conceptualizing Tracking and Other Data-Driven Tech2019-06-20T00:00:00-07:002019-06-20T00:00:00-07:00http://www.markwk.com/qs-mind-map<p>With so many quantified self tools and ways to track a life, it can be a bit confusing. A mind map can help.</p>
<p>A <strong>mind map</strong> is a graphical, visualization technique that is intended to help with structuring, organizing and understanding information as well as facilitate creative thinking. It’s is also one of the best ways to synthesize and understand information in general.</p>
<p>Here’s my <a href="https://github.com/markwk/qs_mind_map">mind map of the quantified self and self-tracking space</a>. The intention is to help you conceptualize the overall tracking technologies space as well as hopefully enable you to better track different aspects of your own life and, in turn, engage with your personal data accordingly.</p>
<p>In the rest of this post, I want to explain the motivation behind the project and briefly walk through why I’ve categorized things accordingly.</p>
<p>One of the principal points I want to make is the division I make between tracking or data collection AND data engagement or being data-driven. While we might obsess about how to track an area, we often fail to take the time to engage with the data we are collecting. For me, one of the key motivations for self-tracking is not data collection but using data to provide a feedback loop towards what I’m trying to understand or a goal I’m trying to reach. This is really only possible if you engage with whatever you are tracking.</p>
<!--more-->
<h3 id="motivation-behind-qs-mind-map">Motivation Behind QS Mind Map</h3>
<p>As someone who has been tracking, writing, thinking and building in the personal data space for several years, I often meet people confused by these ideas but also looking for ways to get started with self-tracking and data-driven self-improvement. I often get asked:</p>
<blockquote>
<p>What should I track? What’s the most beneficial area you’ve found for tracking? What have you learned or changed in your life since tracking? How can I track my time and productive? What should I use to monitor my health or fitness? What wearable do you recommend?</p>
</blockquote>
<p>For anyone getting started with tracking or quantified self, my go-to recommendation is <a href="https://github.com/woop/awesome-quantified-self">Awesome Quantified Self</a>. This is an open source and curated list of the best resources, devices, apps, wearables, and tools for self-tracking (Confession: I’m a contributor and maintainer on this project). For new and experienced alike, this list is a great place to find tools for tracking different areas, like health, time, productivity and more.</p>
<p>Unfortunately the list of option can still be a bit intimidating, especially for a beginner. There are a lot of big categories and multiple tools and apps you might use in each. Choosing the right one can be hard. The list also doesn’t really offer a way to conceptualize and frame the overall quantified self and self-tracking space.</p>
<p>To remedy this, I created my own mind map of quantified self and self-tracking tools and technologies.</p>
<p>You can find the code and complete editable description here: https://github.com/markwk/qs_mind_map.</p>
<p>An interactive and editable version is available at https://coggle.it/diagram/WzGmNxN_zxZw9MbF/t/quantified-self-self-tracking.</p>
<h3 id="a-quantified-self-and-self-tracking-mind-map-three-core-areas">A Quantified Self and Self-Tracking Mind Map: Three Core Areas</h3>
<p>While there are various definitions out there, I define the quantified self and self-tracking as:</p>
<blockquote>
<p>measuring or documenting something about your self to gain meaning or make improvements</p>
</blockquote>
<p>In short the objective is to not just track and collect data but to use data towards goals, like better self-understanding or optimizing your self-improvement. This separation of tracking and engagement lies at a key distinction I made in mind map.</p>
<p>Personally, I divide the quantified self technologies space into three core areas:</p>
<ol>
<li><strong>Tracking Tech</strong>: Health Tracking, Digital Life Tracking, Wearables and Sensors</li>
<li><strong>Data Collection and Data Analysis</strong>: Ways to aggregate and engage with your data.</li>
<li><strong>FutureTech</strong></li>
</ol>
<p>Leaving aside FutureTech, by which I mean a rather open space for biotech, infotech, biohacking, cyborgs, etc., the key point is the difference between the way we track and the way we use that tracking data towards what we want to acheive. In fact, without engagement you really can’t expect much benefit or changes by tracking alone.</p>
<h3 id="remember-data-and-tracking-is-most-effective-with-data-engagement">Remember: Data and tracking is most effective with data engagement</h3>
<p>For me (and my mind map of the quantified self), there is an operational distinction between 1. methods that help us track and collect data and and 2. ways that enable us to engage and use that data.</p>
<p>This distinction is not insignificant since I often hear people complain that having a wearable or using a tracking tool didn’t change their life. There are also academic studies that report to show that wearables were inffective by themselves at changing health outcomes and human behavior. To be honest it’s not really suprising that a wearable alone doesn’t lead to much in the way of health or behavior change. It’s just a tool.</p>
<p>The reality is that wearables and tracking tech doesn’t and likely shouldn’t work without a certain amount of engagement. Much like just buying books on a topic doesn’t result in you becoming an expert or PhD doctorate on that subject, owning a wearable doesn’t magically make you healthier, more productive, etc. A wearable is an opportunity to better understand and improve an area; but you still have to do the work.</p>
<p>For tracking tech to work you need to be engaged. For data to be meaningful, actionable, and life changing, you need to be actively involved.</p>
<p><strong>As such, data and tracking is most effective with data engagement, and a wearable or tracking app is best utilized as a tool to help and support you in pursuit of that goal or life change. Tracking data provides feedback on the area you are trying to understand or improve.</strong></p>
<p>In fact, as a recent study on wearables showed (Stiglbauer, 2019), it’s not whether you had a wearable or not that improved health consciousness, but whether you engaged with that steps data through the app. Health consciousness improved not as a function of having a wearable but in whether you looked at the data in the app.</p>
<p>For me this shows how significant data engagement, data analysis and data visualization can be to an effective data-driven life. While I don’t think or expect that everyone needs to become a data scientist, it’s important to go beyond tracking and engage with your data. Even using a spreadsheet can be a great start.</p>
<p>With that distinction in mind, let’s now look at a few of these branches in more detail.</p>
<h3 id="from-tracking-to-data-engagement-an-outline-of-quantified-self-and-self-tracking-tools">From Tracking to Data Engagement: An Outline of Quantified Self and Self-Tracking Tools</h3>
<p>NOTE: The intention of this section is not to provide a comprehensive list of tools. For that purpose, I recommend you check out <a href="https://github.com/woop/awesome-quantified-self">Awesome Quantified Self</a> on Github. Instead, I will do a high-level walkthrough of going from tracking to data engagement with some links to specific posts I’ve written additional guides on.)</p>
<p>It bares repeating: If you expect self-tracking to help you, you need to be engaged.</p>
<p>Whether you want more objective self-understanding, data-driven personal development or a way to optimize goal pursuits, self-tracking should be coupled with a form of data engagement. For example, this might involve looking at an app that aggregates and contextualizes the data or it might involve collecting the data yourself and looking at it in a separate program or tool.</p>
<h4 id="1-tracking-tech-health-tracking-digital-life-tracking-wearables-and-sensors">1. Tracking Tech: Health Tracking, Digital Life Tracking, Wearables and Sensors</h4>
<p>The options and technologies for the tracking a life is nearly endless, and it seems like almost everyday there is a new way to collect data about yourself. For me the primary way I divide up ways to track is like this:</p>
<h5 id="health-and-fitness-tracking">Health and Fitness Tracking</h5>
<p>These are basically ways to monitor your body. Health tracking can often be separated between ways you track your health status, like medical tests, and methods to measure your health activity and fitness, like activity and sports tracking. Some key areas are:</p>
<ul>
<li>DNA, Genetics, Microbiome: I got my DNA sequenced using 23andMe many years ago and have yet to see much benefit, besides knowing my caffeine metabolism type.</li>
<li>Blood Testing and Biomarkers: I believe blood testing is one of the best metrics of tracking your health status and I’ve written a lot on the topic A good starting point is [<a href="http://www.markwk.com/blood-test-faq.html">Know Thy Blood: Common Questions and Answers about Blood and Blood Testing</a>]](http://www.markwk.com/blood-test-faq.html). I’ve also created an <a href="https://github.com/markwk/awesome-biomarkers">open source directory on blood testing biomarkers</a> too.</li>
<li>Body composition and weight</li>
<li>Heart: It’s easy to track your heart and heart health with a wearable. Personally I’m a huge believer in <a href="http://www.markwk.com/hrv-for-beginners.html">Heart Rate Variablility as a biomarker into my chronic stress and training adaption</a>. I also think the ease of <a href="http://www.markwk.com/blood-pressure-tracking.html">Blood Pressure Tracking</a> makes this something good to check periodically.</li>
<li>Sleep: If I had to recommend a single health area to track with know health and cognitive benefits, it would be sleep. It’s quite easy to track too, and I’ve also written a short <a href="http://www.markwk.com/sleep-tracking-tools.html">sleep tracking guide</a> too.</li>
<li>Diet, Food and Fasting (as well as glucose and metabolism): I find food tracking to be one of the harder areas to track regularly. I’ve done a few experiments with food tracking, especially with just tracking food on a high-level (healthy vs. unhealthy meal, etc.). It appears that fasting or modifying your eating window shows promise for your health too.</li>
<li>Fitness and Activity Tracking: Activity and fitness tracking is arguably the most popular way people track their lives. Whether you use an <a href="http://www.markwk.com/apple-watch-for-self-trackers.html">Apple Watch to Self-Track</a> or some other wearable, running, cycling, sports, lifting and even <a href="http://www.markwk.com/mobility-training-stretching-tracking.html">mobility</a> can all be tracked. Personally I use an <a href="http://www.markwk.com/data-driven-run-training.html">AI coach to manage my marathon run training</a>.</li>
<li>Mood: I’m hopeful that one day we might find a way to automatically detect our moods. I even did a <a href="http://www.markwk.com/mood-tracking-experiment.html">mood tracking experiment</a> myself.</li>
<li>Meditation and Mindfulness: I’m currently experimenting with the Muse Brain Sensing Headband to track and provide real-time feedback on my meditation. It’s also an interesting device to potentially understand my brainwaves too. In the past, I’ve used a number of meditation apps, including <a href="http://www.markwk.com/2016/09/meditation-tracker.html">Calm and Insight Meditation to track my meditation sessions as a habit</a>.</li>
<li>Mind and Cognition: The best tool I’ve found here is quantified-mind.com, which provides a free platform to A/B test your cognitive enhancement self-experiments using research-backed psychology tests of your reaction time and memory.</li>
</ul>
<h5 id="digital-life-tracking">Digital Life Tracking</h5>
<p>For me, basically everything that doesn’t include quantifying your body is covered under the rubic of life tracking. Typically it involves platforms that automatically digitalize aspects of your life, like time, finances, movements, etc. or tools that help you do or organize your work while also provide usage logs and statistics.</p>
<ul>
<li>Time: I’m a huge fan of <a href="http://www.markwk.com/time-tracking-tools.html">time tracking</a> using RescueTime and Toggl as well as keeping a log of my mobile phone screentime and <a href="http://www.markwk.com/youtube-tracking.html">YouTube watching</a>. I wrote <a href="http://www.markwk.com/data-processing-time-tracking.html">a guide on creating a time dashboard with RescueTime, IFTTT and Google Sheets</a>.</li>
<li>Productive Activity: Besides time, I find it’s a good idea (and easy) to track my organizational processes too. This means my tasks, goals, projects, calendar, habits, etc. I’ve used <a href="http://www.markwk.com/task-tracking-with-todoist.html">Todoist to track my tasks</a> for sometime, and it’s a great overall GTD tool with multiple ways to get your data. In terms of <a href="http://www.markwk.com/2016/10/habit-tracking-for-quantified-self.html">habit trackers</a>, I’ve used several over the years, including Habitica. My current habit tracker is Productive on iOS, since it’s simple and reliable. When it comes to <a href="http://www.markwk.com/category/goals-tracking/">goal tracking</a>, there are a lot of different ways you might do this. I currently using <a href="http://www.markwk.com/goal-tracking-with-airtable.html">AirTable to track and manage my goals and long-term projects</a>.</li>
<li>Digital Logs, i.e. Email, Facebook and Google usage logs, etc.</li>
<li>Media Consumption: I track my book reading with Goodreads and Kindle Highlights (<a href="http://www.markwk.com/2016/11/book-reading-tracking.html">here is my write-up</a>). For articles, I use Instapaper and especially like the ability to export my highlights. Last.fm lets me <a href="http://www.markwk.com/2016/09/tracking-music-listening.html">track the soundtrack of my life</a> and I suspect I could even use it to guage my mood too. I log podcast listens using PodcastTracker.com.</li>
<li>Money and finances: I use Mint and a manual spreadsheet to <a href="http://www.markwk.com/tracking-money.html">track my finances</a>.</li>
<li>Location, movement and places, i.e. GPS: I previously loved and used Moves as GPS location logger, but since that service was canceled a few years ago, I no longer have a go-to location and movement tracker.</li>
<li>Tally and life logger tools: For <a href="http://www.markwk.com/2016/10/life-logging-or-tracking-misc.html">lifelogging and ad-hoc tracking experiments</a>, I’ve used tools like Nomie, Hindsight and Reporter. Google Forms is also a good way to track too.</li>
</ul>
<h5 id="wearables-and-sensors">Wearables and Sensors</h5>
<p>Wearables and sensors are essential bridges between our bodies and environments and digital information. Increasingly wearables provide a one-stop shop for digitalizing what goes on in our body, like sleep, movement and even mood. In turn, home sensors help you know about the ambiant conditions around you.</p>
<ul>
<li>Wearables (ex. Fitbit, Miband, Apple Watch, Garmin, Oura Ring)</li>
<li>Environmental Sensors (ex. weather, temperature, air quality, etc.)</li>
</ul>
<p>I currently use an (Apple Watch as my primary Self-Tracking tool)[http://www.markwk.com/apple-watch-for-self-trackers.html]. I’ve had success with Fitbit and especially find it easy to get a reach data set from their API. Xiami MiBand is a great option and cheap, though it isn’t easy to get data from it without sending data first to either Apple Health or Google Fit. Oura Ring appears to be a great and popular option among biohacker, though I’ve found the Oura API to be a bit limited due to how it stores timestamped data.</p>
<p>In any case, most any wearable should work to track activity, steps, heart rate, and sleep. Wearables that use <a href="http://www.markwk.com/science-of-hrv.html">Heart Rate Variability</a> can be great to know more about your health, wellness and stress.</p>
<h4 id="2-data-collection-and-data-analysis">2. Data Collection and Data Analysis</h4>
<p>One of the key aspects of self-tracking that gets neglected is engaging with what you are tracking. While it’s a lot of fun getting a wearable or setting up a new way to track, that’s really just the start of the journey in turning self-tracking into self-transformation.</p>
<p>If it’s a new area, this might involve an initial period of tracking, followed by some data exploration. Oftentimes this will result in setting some kind of goal. You, in turn, use your tracking data to get feedback on your goal pursuit.</p>
<p>Similarly, you might use an array of personal tracking to provide an overview of your health, productive and more. This can then be used in your weekly review or goal tracking.</p>
<p>While there are a lot of ways to collect and do data analysis in the quantified self space, three areas are worth emphasizing:</p>
<ul>
<li><strong>Automation</strong> (i.e. data connectors and integrations): It can be a challenge linking together services and collecting data between services. Fortunately, automation tool IFTTT can be combined with many tracking tools to make it easy to aggregate data. For example, I personally use it to pool my Fitbit, Todoist Tasks and Strava activities into Google Sheets.</li>
<li><strong>Aggregators & Dashboards</strong>: Several apps and websites exist to help you see all of your data into one place, like Gyroscope, BetterSelf or Exist.io. There are also open source tools and code, like <a href="https://github.com/markwk/qs_ledger">QS Ledger</a>, can facilitating data collection and data visualization. For example, I used QS Ledger to create <a href="http://www.markwk.com2019/01/year-in-data.html">my year in data</a>.</li>
<li><strong>Data-Driven Tools and Bots</strong>: There are a few tools today that take tracking data and apply it to actual advice and engagement. For example, Lark is an activity chatbot and TrainAsOne is an AI running plan coach. As technology advances, I expect to see more tools and services that leverage data towards targetted coaching and personalized health.</li>
</ul>
<p>Beyond these cookie-cutter approaches, there is a lot of power and potential in do-it-yourself solutions. You don’t need to be a programmer or data scientists to do it either. IFTTT can be used to pull data into Google Sheets from different tracking services.</p>
<p>You can then use a spreadsheet app to create a simple dashboards or use more advanced data dashboard tools like Google Data Studio or Tableau. Furthermore, if you have basic python data science skills, my open source project <a href="https://github.com/markwk/qs_ledger">QS Ledger</a> provides a number of starter notebooks for data collection and visualization too.</p>
<p>Whatever method you choose, if you want to make your self-tracking most effective, find a way to regularly engage with your data. It’s arguably the best way to transform self-tracking into data-driven self-improvement.</p>
<h4 id="3-futuretech">3. FutureTech</h4>
<p>While I think it’s a topic left for a future post, in my conception of the quantified self space I leave open a third core thread for FutureTech. For me, this is where science fiction meets reality. I separate this from existing data tracking tech and data analysis tools, because it is in many ways a fusion of the two and a new thing onto itself.</p>
<p>Some examples might include Biotech / Infotech, New Sensors on the body and in the environment, Neurofeedback, New Data Usage (like using Siri or Alexa data to detect mood), Shared and Collaborative Data (like taking group data to provide collective insights and trends), Biosensing (tools that create new sensory awareness) and Biohacking (where you use data and human augmentation tech to go beyond current human capacities).</p>
<p>Overall, this is an exciting area but still too early to say what it will become. Thus, I’m calling it FutureTech!</p>
<h2 id="conclusion-ask-a-question-find-a-tracker-and-engage">Conclusion: Ask a Question, Find a Tracker, and Engage</h2>
<p>In this post we looked at my organization and conceptual of the Quantified Self space using a mind map. For me the key distinction is between tools to track and collect data and ways to engage and use that data.</p>
<p>For me, the core of the quantified self movement and idea is nothing new. It goes back to early Greek thinkers and philosophers and the quest to “know thy self.” The major change is we have tools and technologies that help us to do that in new and interesting ways.</p>
<p>Personally, I find the most beneficial way to use these tools and technologies is through an on-going process. First, I find or ask a question. I then do research and use tracking tools to collect some data. After awhile I then look at my data to better understand myself. Where am I at? What and how am I do? Finally, I set a goal and start a process of self-improvement with data-driven feedback. Throughout the process I engage and learn and, ultimately grow.</p>
<p><br /></p>
<p>Best of luck on your journey and happy tracking!</p>
<p><br /></p>
<h5 id="sources">Sources:</h5>
<ul>
<li>Quantified Self Mind Map, available online at https://github.com/markwk/qs_mind_map.</li>
<li>Awesome Quantified Self, available online at https://github.com/woop/awesome-quantified-self.</li>
<li>Stiglbauer, B., Weber, S., & Batinic, B. (2019). Does your health really benefit from using a self-tracking device? Evidence from a longitudinal randomized control trial. Computers in Human Behavior.</li>
</ul>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}With so many quantified self tools and ways to track a life, it can be a bit confusing. A mind map can help.Goal Tracker for AirTable: A Flexible Tool for Goal Pursuit Tracking and Management2019-03-19T00:00:00-07:002019-03-19T00:00:00-07:00http://www.markwk.com/goal-tracker-for-airtable<p><img src="/generated/images/2019-resources/goal-tracker-cover-800-dd9e40d3d.jpg" srcset="/generated/images/2019-resources/goal-tracker-cover-400-dd9e40d3d.jpg 400w, /generated/images/2019-resources/goal-tracker-cover-600-dd9e40d3d.jpg 600w, /generated/images/2019-resources/goal-tracker-cover-800-dd9e40d3d.jpg 800w, /generated/images/2019-resources/goal-tracker-cover-1000-dd9e40d3d.jpg 1000w" /></p>
<p>Having a goal is easy and exciting. But goals only get accomplished when you pursue them, not because you simply make them. So, if you want to improve your goal attainment, arguably one of the most important goal action steps you can do is to convert your goal intentions into goal pursuits.</p>
<p><em>What is a goal pursuit and how to track it?</em></p>
<p>I define a goal pursuit like this: <strong>Goal Pursuit = Goal + Time Period + Specifics</strong></p>
<p>Obviously, there is more to this formula and to pursuing goals, but this gets us to a few of the essentials when it comes to both the science of goals and how to think about organizing and managing your goals over time. It also gets to a framework that allows for tracking your goal pursuits too.</p>
<p>Let’s break this down a bit:</p>
<ul>
<li>A <strong>goal or goal intention</strong> is the thing want. It might be quite specific (“Run a marathon under 4 hours”), or it might be quite abstract (“Become a recognized expert in XYZ domain”). But generally a goal is an object of desire that lacks the defined steps on how you will get there. It’s the fantasy target.</li>
<li>By contrast, a <strong>goal pursuit</strong> represents a state change. It’s no longer just an idea or object of desire; it is a thing you are working on.</li>
<li>The <strong>time period</strong> just means we’ve stated an expansive of time when we will pursue that goal. Even better is if you state a habitual time and place.</li>
<li>The <strong>specifics</strong> of your goals means you’ve stated exactly what you plan to do to move towards that goal and ideally made it something you can measure. Being more specific when you set a goal is one of the key activities to improving the success and performance of your goals</li>
</ul>
<p>Practically-speaking a goal pursuit takes form once you take your abstract goal and get specific about how you’ll get there. Scientists call this the “goal decision” or “goal striving.” Intuitively there is something different between wanting something abstractly and actually doing it. This is the difference between a goal fantasy and goal pursuit.</p>
<p>The science of goals pushes us both to think about goals and goal pursuits as multiple step endevours and to apply science-backed strategies at key junction in our striving. According to the Goal Setting Theory, the best technique in a goal pursuit is <em>get specific</em> and set an exact, targetted, measurable and difficult initiative to work on. According to the well-research technique of implementation intentions or if-then plans, one effective goal practice is to set the <em>how</em>, the <em>when</em>, and the <em>where</em> so you figure out a cue or trigger, like a place or time and then mentally link it to the goal behavior you want to induce.</p>
<p><strong>How might we go about organizing, managing, and even tracking our goals and goal pursuits?</strong></p>
<p>As a long-time self-tracker and proponent of the data-driven life, goals have proven a rather elusive area to track. While it’s relatively easy to track many aspects of our lives from books read and tasks completed to miles run and time on devices, tracking goals proves difficult for two reasons:</p>
<p>First, we often think of goals as just a list of things we want, so it feels like there isn’t much to track. Just make a list and check them off when you complete them.</p>
<p>Second, when we do attempt to track our goals, we often limit our thinking about tracking goals to something measureable. We come up with some related output of our goals, like miles run, time, habits, etc. A tool like BeeMinder is a good example of a tool that tracks an underlying output metric and helps us view our goal progress accordingly. While this can be a good technique for some goals, it actually isn’t tracking goals as a process or cycle of action steps. What we want is a goal tracker that tracks much more than a single metric.</p>
<p>Goals are more than just a list of wishes we hope to get and more than a single definable metric. Goals exist the nexus of our goal intentions and specific goal pursuits over time. Goals often go through a cycle of setting, pursuing and reviewing. There are steps forward and steps back and a lot of figuring stuff out. What we want to track and manage is this long-term cycle of goal pursuits over time. And, as far as I know, there isn’t a complete tool available today that lets us manage and track our goals as multi-stage pursuits.</p>
<p>So in order to solve this problem and apply techniques from the science of goals, I’m excited to share a tool I’ve built to help you better track and manage goals.</p>
<p>Using Airtable, which is a mix of a spreadsheet application and a database, I’ve created a goal tracker that allows me to keep an on-going list of my goals and dreams and to convert those into periodic goal pursuits that I further track, manage, and optimize. Additionally, the flexiblity of AirTable allows you to apply some of the most actionable and effective techniques from the science of goals. This means we are not only tracking what’s happening but we are applying powerful strategies to get better at reaching our goals.</p>
<p>In short, <strong>Goal Tracker for AirTable is a free tool that helps you organize, manage and track both your goals list and your goal pursuits</strong>.</p>
<p>Goal Tracker provides a simple way to record all of your current, past and future goals. You then use these goals to formulate goal pursuits. You can think of goal pursuits as a short-term goal, initiative or project. Goal pursuits link those goals to a time period and allow you to get specific about what you’ll do. For example, if your goal is “run a marathon,” then your goal pursuit might be “May 2018 (Run Marathon): Complete 10 training runs, run 100km, and complete a half marathon.”</p>
<p>Using the tool, you now have a list of goal pursuits so you can better organize and visualize over time. Goals are a struggle and will fail, but having a tool to organize and track allows you better stay on top of the process. You are not only pursuing a goal but also optimizng the meta-skill of managing goals. By consistently linking goals with a specific initiative in a certain time frame, you have effectively applied several key techniques from the science of goals too.</p>
<center><a href="https://airtable.com/shrIu55ZjWgNUyZTz" target="_blank" class="button">Get Goal Tracker for AirTable Now</a></center>
<p><br /></p>
<p><a href="https://airtable.com/shrIu55ZjWgNUyZTz">Goal Tracker</a> is a free tool you can download for AirTable. I have no affiliation with AirTable. I just think it’s a cool tool that works well for flexible information management and for prototyping goal and project management tools like these. Feel free to customize, add or tweak and, of course, send me your thoughts, feedback and areas to improve! Additionally, if you prefer another tool, I think you can accomplish the same style of goal tracking using just simple files or another spreadsheet app.</p>
<p>In the rest of post, we are going look at the core features and usage of Goal Tracker for Airtable. We will first briefly look at AirTable, what it is and how it works. After that, our main objective is to get you up and running with using the tool to track and manage our goals. Along the way, I’ve included a few examples of customizations and deeper concepts related to how AirTable works and how it relates to both the science of goals and productivity principles for good goal management. Finally, I’ll conclude by sharing my own motivation for building this tool and tips for the overly ambitious goal pursuer been using it.</p>
<p><br /></p>
<p><em>Note:</em> The psychology of goals lies at the heart of why I’ve been building and using this tool. I’ve purposefully left out too many academic references here to focus on the tool and usage itself. If you are interesting in exploring the underlying science, check out my on-going blog series on the <a href="http://www.markwk.com/category/science-of-goals/">Science of Goals</a>.</p>
<!--more-->
<h2 id="what-is-airtable">What is AirTable?</h2>
<p><img src="https://airtable.com/images/home/grid_view.png" alt="Screenshot of AirTable" /></p>
<p>Sometimes you need a tool that looks and acts like a spreadsheet but is more powerful and acts like a database. That’s the core of what AirTable does. It provides a familar user experience from spreadsheets, but additionally, you get a few more sophisticated ways for information management and for advanced functionalites.</p>
<p>While AirTable does have some paid features and a space usage limit, I’ve found the free version perfectly suited for my needs so far.</p>
<p>AirTable provides a way to create a range of rows and columns, much like you would with any spreadsheet. The columns are actually fields much like in a database. You can have fields or columns for different data types (text, numbers, selection, list, etc.).</p>
<p>Importantly, you are also able link together multiple tables into pretty complex structures. This means you can have use something like reference tables. This allows for dynamic fields, do ingcalculations and lookups, and forming aggregation tables.</p>
<p>This makes it possible to organize any type of information and to create specific tools to manage that information through tailored processes and workflows. For example, for me, it works great as a CRM or customized project manager among other things.</p>
<p>A complete introduction to AirTable is outside the scope of this post. If you need help, their site has tons of blog posts and tutorials as well as several useful examples to you can clone and start using.</p>
<p>When it comes to tracking and managing goals, AirTable provides exactly what we need. We are able to create multiple tables for our main areas (goals, time periods and goal pursuits) and then link them together to create a goal management workflow. Our goal structure ends up looking like this:</p>
<p><img src="/generated/images/2019-resources/goal-pursuit-structure-800-1fe990401.png" srcset="/generated/images/2019-resources/goal-pursuit-structure-400-1fe990401.png 400w, /generated/images/2019-resources/goal-pursuit-structure-600-1fe990401.png 600w, /generated/images/2019-resources/goal-pursuit-structure-800-1fe990401.png 800w, /generated/images/2019-resources/goal-pursuit-structure-1000-1fe990401.png 1000w" /></p>
<p>We have a table dedicated for goals and another for time periods. These then get linked and more specific when we create goal pursuits.</p>
<p>In a technical sense, what we are doing is using two tables (goals and time period) and creating a third table (goal pursuits) to manage the relationship between them. These are call join or junction tables, and they allow us to track aspects of that thing-in-between, i.e. our goal pursuits. If you are confused or interested in understanding this concept, check out AirTable’s <a href="https://support.airtable.com/hc/en-us/articles/218734758-A-beginner-s-guide-to-many-to-many-relationships">A beginner’s guide to many-to-many relationships</a>.</p>
<h2 id="goal-tracker-for-airtable-how-it-works">Goal Tracker for AirTable: How It Works</h2>
<p>There are a lot of tools out there that help you track different aspects of your life, like tasks, time, projects and various health metrics. I’ve used and talked a lot about them in my attempt at <a href="">tracking everything</a>. While there are some that explicitly claim to track goals, like Beeminder, the reality is that they only track certain outcomes towards a goal, like kilometers run, pages read, etc. None of them track the goal pursuit process.</p>
<p>Goals require multiple projects, tasks, and pursuits as we advance towards them. Most goals take time and involve several action stages. If we want to track that process and even better improve our pursuit, then we need a tool that helps use better organize and track goals as a process, rather than just a static list of dreams. For that reason, I created Goal Tracker for AirTable.</p>
<p>The primary aim of Goal Tracker is to help you track the organizational aspects of your goals and goal pursuits. You keep a list of your goals and formulate goal pursuits from that list. You can then add and update additional information related to them over that time period.</p>
<p>To quickly summarize, Goal Tracker is divided into three core sections or tables:</p>
<ul>
<li><strong>Goals & Dreams</strong>: This where you store your typical idea of a goal or dream. You note what it is and can add a description and category, if you like. We use this table as a reference in our goal pursuits.</li>
<li><strong>Time Periods</strong>: This is a simple calendar-like structure, which provides the brackets of time in which you plan to work on a goal.</li>
<li><strong>Goal Pursuits</strong>: This where the goal action happens. A goal becomes a goal pursuit when you assign it to a time period, get specific, and start working on your goal.</li>
</ul>
<p>A good, science-backed goal pursuit is specific and measurable. You state what you’ll work on and when, where and how you’ll work on it. Goal Tracker is all about helping do just that.</p>
<h4 id="goals--dreams-list">Goals & Dreams List</h4>
<p><img src="/generated/images/2019-resources/goal-tracker-example-01-800-cc2dd74f9.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-01-400-cc2dd74f9.jpg 400w, /generated/images/2019-resources/goal-tracker-example-01-600-cc2dd74f9.jpg 600w, /generated/images/2019-resources/goal-tracker-example-01-800-cc2dd74f9.jpg 800w" /></p>
<p>When it comes to Goal Tracker, the Goal and Dreams list should <em>NOT</em> be considered its primary functionality. This is just a place where you store a list of all possible things you might work on.</p>
<p>You are really just storing your goal intentions. These are typically abstract targets of desire. They might be dreams and fantasies we hope to one day acheive, like write a novel or travel to Paris or they could our identity (Writer, Teacher, etc.). It might also be a defined purpose or personal mission. They might maybe-things and actual things we are engaging in.</p>
<p>While often conflated, it’s important to distinguish between a goal intention and goal pursuit. A goal intention is the thing want, while a goal pursuit is our cumulative actions and plans towards attaining that goal during a certain time period. When it comes to tracking and organizing your goals, this distinction is critical, because if you tell yourself you have a goal but never pursue it, then it’s just wishful thinking.</p>
<p>Beyond just keeping a list, you can use AirTable to help you organize and prioritize your high-level goals. For example, you can use <em>Buffet’s 5/25 rule</em> to come up with goal areas of your life to prioritize and areas you need to ignore. Additionally, you can use categories to see if you have the balance you want.</p>
<p>Most likely you already have a list of goals, so transfer some of them into the system. As you come up with new goals or need more sorting, return to this list. Remember the point is not to obcess about your goals but to use these goal intentions as the starting point for formulating specific goal pursuits.</p>
<h4 id="time-frames--when">Time Frames = When</h4>
<p><img src="/generated/images/2019-resources/goal-tracker-example-02-800-748a866d4.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-02-400-748a866d4.jpg 400w, /generated/images/2019-resources/goal-tracker-example-02-600-748a866d4.jpg 600w, /generated/images/2019-resources/goal-tracker-example-02-800-748a866d4.jpg 800w" /></p>
<p>One big reason why many goals fail is because you never defined the <em>when</em>. You simply stated a goal intention but haven’t provided a specific when, where and how. This problem is most accute with yearly goals, since we can continually tell ourselves later this year until we reach Dec 31st! So, while it’s okay to have yearly goals, it’s better to define a goal pursuit on shorter time frame that forces you into action and allows less procrastination.</p>
<p>Goal Tracker helps you by ensuring all goal pursuits define a time frame. You basically need to include a time frame if you want to create a goal pursuit.</p>
<p>Time periods in Goal Tracker are flexible. They might be weekly, monthly, quarterly, yearly or anything you like. I tend to focus on monthly and quarterly goals, so in the template I’ve provided examples of those.</p>
<p>In order add your own just creat new item or add a row in the bottom. Add a clear name and then include a start and finish date. After that, you can then assign it to a goal pursuit.</p>
<h4 id="goal-pursuit--goal--time-period--specifics">Goal Pursuit = Goal + Time Period + Specifics</h4>
<p><img src="/generated/images/2019-resources/goal-tracker-example-03-800-7ec0c911c.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-03-400-7ec0c911c.jpg 400w, /generated/images/2019-resources/goal-tracker-example-03-600-7ec0c911c.jpg 600w, /generated/images/2019-resources/goal-tracker-example-03-800-7ec0c911c.jpg 800w" /></p>
<p>Goal Tracker for AirTable is designed around helping you focus on goal pursuits.</p>
<p>One of the reasons why many goals fail is because they existed only as a goal intention. It was something you may have wanted but not enough to actual pursue it. I’m not even sure it’s fair to call them goal failures, since you never really worked on them.</p>
<p>This is why in the science of goals we make a distinction between a goal intention and goal pursuit. For anyone working on their goals, your main objective should be your goal pursuits.</p>
<p>A goal pursuit involves taking one of your goals, assigning it a time period and deciding on the specifics involved. You can create goal pursuits by either adding a new row in the table and filling in the differnet parts or using the special form provided here:</p>
<p><img src="/generated/images/2019-resources/goal-tracker-example-04-450-a80aa7d44.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-04-400-a80aa7d44.jpg 400w, /generated/images/2019-resources/goal-tracker-example-04-450-a80aa7d44.jpg 450w" /></p>
<p>The form is not necessary and merely provides a bit more context and help about the different fields and the science behind them.</p>
<p>Now that we understand how Goal Tracker for AirTable works, I’d like to share a few aspects on how to use it.</p>
<h2 id="goal-tracker-for-airtable-how-to-use-it">Goal Tracker for AirTable: How to Use It</h2>
<p>To reiterate, the primary aim of Goal Tracker is to help you track the organizational aspects of your goal pursuits, like which goals you are working on, assigning time periods and recording various properties related to them.</p>
<p>There are a lot of different ways you might use a tool like this. AirTable makes it really easy to modify it to how you work best or a specific aspect of goal techniques you want to apply and track.</p>
<h3 id="key-goal-actions">Key Goal Actions</h3>
<p>There are various aspects and stages to attaining a goal. A certain part of many goals might be habits and tasks. For these, I recommend using a <a href="#">habit tracker</a> or <a href="#">task manager</a>. So, besides the actual goal actions, here are a few key organizational actions for your goals:</p>
<h5 id="1-capture">1. Capture:</h5>
<p><em>What are your wishes, dreams and desires?</em></p>
<p>These are your goal intentions or more commonly just called your “goals.” Record them in the goal and dreams table.</p>
<p>In <a href="http://www.markwk.com/category/getting-things-done">GTD</a> parlance, do a brain dump of your goals. Additionally feel free to brain storm and add categories and goal pursuits too. You can organize them after.</p>
<h5 id="2-deliberate-and-decide">2. Deliberate and Decide:</h5>
<p><em>Do you really want to pursue that goal? Is it desirable? Is it feasible? And why</em></p>
<p>This is the “because” or motivation behind your goal pursuit.</p>
<p>There is a good amount of research showing how critical motivation is to successful goal pursuits. One powerful technique <em>mental contrasting</em> which can be useful in this predecision, pre-action phase since it forces you to figure out both the drive behind the goal and obstacles in the way. You don’t necessarily have to formally do this inside the tool, but a field has been provided so you can.</p>
<h5 id="3-formulate-and-plan">3. Formulate and plan:</h5>
<p><em>What is the specific thing you will do? How, when and where will you do it?</em></p>
<p>Once you decide on a goal, it becomes a goal pursuit. The key task then is figuring out specifically what you are going to do. There is a required and central field here to note the specifics of your goal pursuit. It could be a one-liner or it could be a few points.</p>
<p>Goal Setting Theory recommends making it specific and difficult. Another good technique is using Implementation Intentions or “If-then” plans. This is process of choosing a trigger, like “When I open my computer first thing in the morning…”, and deciding the action you’ll take, like “I’ll write a short journal entry of recent ideas, memories or things I’m grateful for.”</p>
<p>Ultimately you aim to plan habits, initiatives and goal-striving actions.</p>
<h5 id="4-manage-and-schedule">4. Manage and schedule:</h5>
<p><em>When are you will you work on this goal? Which time period, month or quarter? For a habit, which time of day?</em></p>
<p>For all pratical purposes, a goal pursuit is something you are planning or acting upon during a certain time period. So, one key action I’ve learned around managing my goals is scheduling that goal pursuit within a certain amount of time. This act of scheduling forces me to figure out blocks of time that will allow me to advance that goal. If I can’t find the time or have scheduled too many pursuits in one period, that goal pursuit will fail or perform poorly.</p>
<p>Goal Tracker makes it pretty to visualize just how many goals you have scheduled in some time period. You can look back in the past to see what balance worked for goal attainment. You can then adjust current goal pursuits and even move items into a different future time period.</p>
<p><img src="/generated/images/2019-resources/goal-tracker-example-05-800-9e2d38534.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-05-400-9e2d38534.jpg 400w, /generated/images/2019-resources/goal-tracker-example-05-600-9e2d38534.jpg 600w, /generated/images/2019-resources/goal-tracker-example-05-800-9e2d38534.jpg 800w" /></p>
<p>The point is to find the right balance of things to work on in the short-term but in view of necessary steps you can work on later.</p>
<h5 id="5-track-it-manage-it">5. Track It, Manage It:</h5>
<p><em>Am I confident in my goal pursuit? Where do my goals stand? Is there a measurable? What’s my target and current number? How much progress have I made?</em></p>
<p>These are a few questions you might use to track a specific goal pursuit. There are many others and you can easily add or modify according to how you want to do it. For now, I’ve included a modified technique from Google’s OKR goal system to track your goal pursuits. These provide a good starting point since it measures both the confidence you have and some way to gauge progress.</p>
<p>Here are the fields:</p>
<ul>
<li><strong>Goal Confidence</strong>: How confident are you of goal pursuit success? Green = All good, continue on. Yellow = There is an issue and something needs looking into. Red = Failing or Failed.</li>
<li><strong>Goal Status</strong>: This is a simple field to mark if a goal is active, completed or some other project management status.</li>
<li><strong>Goal Metric</strong>: If the goal pursuit can be tracked with a number, then the goal metric or measurable is the thing you’d be measuring, like Blog Posts Published, Miles Run, Sales Numbers, Money Saved, etc.</li>
<li><strong>Goal Target</strong>: This is the target number of your mesurable. For example, 50 km, 3 blog posts, $1000 savings, etc.</li>
<li><strong>Current Number:</strong> This is a simple way to note your current measurable numer.</li>
<li><strong>Progress Percentage</strong>: This is a way to track your progress. If the goal is meaurable, then this your current number divided by your target. If it isn’t really possible to measure, then just subjectively note how close you are as a percentage.</li>
</ul>
<p>These fields provide a solid first step towards goal tracking as a pursuit. They let you note both how subjectively confident you are about a goal and a few ways to objectively measure your progress.</p>
<p>Depending on the goal pursuit, tracking might be something you do regularly or you might just do it periodically along with your weekly or monthly review.</p>
<p>Since our focus is on tracking the goal process rather than a single metric, it isn’t the most sophisticated system. This is one area for possible future improvement of Goal Tracker so we can update our progress using data collected from external trackers.</p>
<h5 id="6-review">6. Review</h5>
<p><em>What goals did I work on? How did they go? What did I learn?</em></p>
<p><img src="/generated/images/2019-resources/goal-tracker-example-06-800-d1ace8cc7.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-06-400-d1ace8cc7.jpg 400w, /generated/images/2019-resources/goal-tracker-example-06-600-d1ace8cc7.jpg 600w, /generated/images/2019-resources/goal-tracker-example-06-800-d1ace8cc7.jpg 800w" /></p>
<p>Goals can and should come to some form of endings. During these ends, it is helpful to take a look back to review and evaluate how those goals went.</p>
<p>I’m a big beliver in the power of consistent reviews, especially <a href="http://www.markwk.com/data-driven-weekly-review.html">weekly reviews</a>. Reviews allow you to check-in and reflect on how a certain period went. They allow you take a step back and see your high-level process worked, and then think forward towards what you might change or do better in the future. It also can be a stress-reliever.</p>
<p>When it comes to goals and goal pursuits, the review and evaluation phase can be used to help you learn a few things about you and your goals.</p>
<p>The first thing it helps with is just giving yourself a chance to review how well you did in pursuit of that goal over a certain period. Did you reach your goal? How did it go? This reflection should combine both postive aspects, like lessons learned and what you are most proud of, and negative aspects, like what went wrong or ways you might improve going foward.</p>
<p>The second thing a goal review does is it allows you to transition from one goal pursuit to the next. While goals might take months or years to reach, goal pursuit research indicates the importance of avoiding goals that are too distal, i.e. to far in the future. Instead, the best strategy is to divide up that pursuit into a series of “missions” or initiatives over time. This is why Goal Tracker emphasizes goal pursuits segmented into time periods.</p>
<p>By breaking down your abstract goals in sub-goals and specific projects, you can manage and develop them over multiple time periods. You build up a chain of positive steps and and develop skills towards getting to that goal.</p>
<p>In Goal Tracker for Airtable, I’ve provided a separate view for past goal pursuits:</p>
<p><img src="/generated/images/2019-resources/goal-tracker-example-07-800-0fd246c91.jpg" srcset="/generated/images/2019-resources/goal-tracker-example-07-400-0fd246c91.jpg 400w, /generated/images/2019-resources/goal-tracker-example-07-600-0fd246c91.jpg 600w, /generated/images/2019-resources/goal-tracker-example-07-800-0fd246c91.jpg 800w" /></p>
<p>This breakdown visualizes goal pursuits in reverse chronologial order and you can add your reflections and notes accordingly. You might do them in the tool using a dedicated field or elsewhere in a separate notebook or notes tool.</p>
<p>I recommend scheduing periodic check-in’s to track progress and review your goals. Personally, each year I spend some time setting my yearly goals and I then attempt to translate them into monthly or quarterly objectives. For some goals, I schedule them later in the year. I mostly work and review my work on a weekly basis. I set 1-2 big project objectives each week and 2-6 smaller step goals too. I then do roughly a 3-6 week goal cycles where I review and log my progress and I revise my goal pursuits.</p>
<p>Ultimately you’ll never acheive every goal you set but hopefully by using a tool like this you can better align, organize and schedule what matters and set specific objectives across time. In turn, you get a clear log of what you did and when. You know which goals fell through and which progressed. For me that’s the power of having a goal tracking tool like this.</p>
<h2 id="conclusion-goal-organization-management-and-tracking-for-the-overly-ambitious">Conclusion: Goal Organization, Management and Tracking for the Overly Ambitious</h2>
<p>Goal Tracker provides a focused tool for keeping and prioritizing a list of your goals and dream and for converting those fantasies into action-oriented goal pursuits.</p>
<p>In this post, we did a pretty high-level look at the tool I’ve built using AirTable. Its primary purpose is to help you organize and track multiple goal pursuits over time. By splitting goal intentions from goal pursuits, you get better at focusing on the later while keeping a sorted list of the form. In turn, having a log of your goal pursuits allows you to apply key techniques at different stages from the science of goal. This gives you a record of your goal pursuit history and allows you to improve your efforts at pursuing and obtaining your goals.</p>
<p>The tool is available for free and you can download it now here:</p>
<center><a href="https://airtable.com/shrIu55ZjWgNUyZTz" target="_blank" class="button">Goal Tracker for AirTable</a></center>
<p><br /></p>
<p>It’s also fully customizable so you can modify and augment to fit your needs and apply what you find works best in pursuit of managing and reaching your objectives.</p>
<p>To conclude, I want to talk a bit about the motivation behind why I build this tool and how I personally use it and the scientific principles behind it.</p>
<p>Arguably the best strategy when it comes to goal pursuits is radical and complete focus. Have one singular goal, maybe two, and dedicate as much of your energy, time and self into it. In this goal pursuit situation of a singular goal and singular focus, you can largely forget about goal organization or management. Just focus on completing the next best actionable items and acheiving desireable outcomes that move your goal forward. Work until that goal is complete, and then find a new goal. Rinse and repeat.</p>
<p>Unfortunately, the reality is that many (myself included) have and will have multiple goals at the same time and across our lives. We all know it’s better if you stick to one or two things, but as Robert Heinlein put it a famous <a href="https://en.wikipedia.org/wiki/Competent_man">quote</a>, “Specialization is for insects.” I believe a happy, passionate and commited human life is one where we all have and should have lots of goals, both conscious and unconscious. We are overly ambitious creatures. We have big dreams, goals and things we want to reach.</p>
<p>Having multiple goals and goal pursuits over time is one of the key assumptions I applied to building this tool. The tool is built to allow you to express and capture each and every goal you might have. In turn, it helps you to think about and transform the goals that do matter into action-oriented goal pursuits. For me this transformation of goals into goal pursuits is crucial if we want to improve our goal processes.</p>
<p>Many will fail and that’s ok, but a few will succeed as we discover something we believe in truly and, with a bit of luck, something that works out successfully. But to get to those driving goals and passions is a process of trial and error over time. We need to approach our goals and search for purposes as a multiple stage process involving various goal pursuits over time.</p>
<p>Goal Tracker is largely my expression and attempt at organzing, managing, and tracking the pursuit of multiple goals over an infinite time frame of a human life. Being overly ambitious is okay, but it’s better if we use a tool to help us undrstand our goals better and get more of these goals accomplished. This is especially true for complex goals involving a long time frame and ambigity about is the next best thing to do.</p>
<p>Hopefully Goal Tracker can help you in reaching your short-term goal pursuits and driving at your deeper life purposes. It’s certainly helped me in mine.</p>
<p><br /></p>
<p>Best wishes and happy tracking!</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}Science of Goals: Goals as a Multi-Stage Pursuit2019-03-08T00:00:00-08:002019-03-08T00:00:00-08:00http://www.markwk.com/science-of-goals-2<p>When it comes to goals, we commonly conflate several different actions and phenomonon under this singular term of “goal.” But this one term hides an important range of dimensions and actions we undertake when we think about and endevour to reach our goals. One of the most crucial differences is between <em>a goal as an intention</em> and <em>a goal as a pursuit</em>.</p>
<p>German-American psychologist Kurt Lewin was one of the earliest to make the distinction between what he called <strong>goal setting</strong> and <strong>goal striving</strong> (Lewin, 1926). His point being that we have a period where we deliberate, ponder and possibly select our goals; and a period where we actually strive towards realizing those goals.</p>
<p>In our own lives we all typically have a huge list of things we want to accomplish in both the short-term and long-term. For example, write a novel, learn French, take a trip to Australia, buy a house, read Homo Sapiens, etc. The items that we expect to take a considerable amount of time or effort we often call “dreams” or “long-term goals,” while the shorter term stuff that take a few days or weeks we might refer to as projects, initiatives or short-term goals. Usage of the term “goal” here proves slippery. Is it a term for our hopes, dreams and aspirations or is it a word to convey something we are actually pursuing? What’s going on when we talk about goals?</p>
<p>In the last couple decades, a considerable amount of research by psychologists has gone into understanding goals, what they are and the different actions we must take to attain them. There are also increasingly a number of powerful strategies that can leverage to improve our goals too.</p>
<p>While much of the early focus in the research on goals was placed on goal setting and the “content” of our goals (a topic we explored in <a href="http://www.markwk.com/science-of-goals.html">goal setting for improving task performance</a>), acheiving our goals goes beyond just setting a goal. Acheiving a goal isn’t easy. A significant part of the challenge comes from what psychology calls “self-regulation,” by which I mean individuals must manage an ambiguous process, make decisions and take actions regarding allocation of resources and effort across time and varying situations.</p>
<p>You don’t just set a goal, and the magic of reaching that goal happens. Things need to be figured out. Much like Joseph Cambell’s Hero with a Thousand Faces, we embark on a journey with obstacles and challeges at different points in order to reach our objective. In short, <strong>goals exist as a multistage pursuit</strong>.</p>
<p>Psychologists now recognize this much more complex and nuanced nature of our goal pursuit, in particular this aspect of self-regulation and different stages. According to the Model of Action Stages, which we will explore in depth below, psychologists have identified four action phases involved when we strive towards a goal, namely: deliberation, planning, action and evalution.</p>
<p>Researchers have shown that there also exists a critical transition between the predecision phase of goal deliberation (e.g. what goals should I pursue and why?) and our post-decision when have commited to a goal (e.g. how can I achieve this goal and what do I need to do?). Metaphorically, they call this “crossing the Rubicon” in reference to Julius Cesaer’s overthrowing of Rome, and it refers to a recognizable shift in our mindset or psychological orientation. Pre-decision or pre-goal commitment, we deliberate and consider. Post-decision, we plan and take actions.</p>
<p>We see this in our own lives. We often have a big list of dreams and fantasies, but a much shorter list of active pursuits. In-between our goals typically have specific phases and cycles they go through, including planning, execution and evalution.</p>
<p>Goal are a multiple stage pursuit. I believe the science of goals can help us improve how we understand and how we reach our goals. Using the Rubicon Model of Action Stages, we get a “big picture” idea of typical goal pursuits and can start to recognize where we are at. By knowing the stage we are at with a certain goal, we can realize the key tasks and challenges we face. We then can apply the right mindset to deal them. Finally, by learning and deploying the right approaches and optimal strategies, we can get better both at abtaining our goals and at perceiving when to let certain goals fail.</p>
<p>Let’s get started looking at the science goals and the stages involved in a goal pursuit!</p>
<p><br /></p>
<p><em>NOTE 1</em>: This post is part of a series on the <a href="http://www.markwk.com/category/science-of-goals">science of goals</a>. Also check out <a href="http://www.markwk.com/science-of-goals.html">Goal Setting as a Key Influence on Performance</a>.</p>
<!--more-->
<h2 id="what-is-a-goal-from-goals-as-an-intention-to-goals-as-a-pursuit">What is a Goal? From Goals as an Intention to Goals as a Pursuit</h2>
<p><img src="/generated/images/2019-resources/goal-without-a-plan-450-f2ea43833.png" srcset="/generated/images/2019-resources/goal-without-a-plan-400-f2ea43833.png 400w, /generated/images/2019-resources/goal-without-a-plan-450-f2ea43833.png 450w" /></p>
<p>The term “goal” can be defined in a few ways, and the conflation of these meanings into a single term not only causes confusion when we want to take about the science of goals, but it also causes us as goal pursuers to confuse what we are doing when we pursue goals.</p>
<p>For example, in my life, I use the term “goal” in two markedly different situations. On the one hand, I have this huge list of goals that I hope to one day do, try or reach. For example, visit all 7 continents, learn French, write a novel, etc. In a scientific sense it is misleading to call these goals or goal pursuits, especially when I’m not taking action towards their abtainment. It’s better to call them dreams or fantasies.</p>
<p>On the other hand, I do have several goal pursuits I work on at anytime. These are active things I’m striving to advance and reach. While it might be okay to have an infinite list of dream goals, we are limited to how many active goal pursuits one can effectively maintain. Combining too many projects with limited time, focus and resources is assuredly a recipe forgoal failure.</p>
<p><em>So, what is a goal?</em></p>
<p>At its simplest, a goal is the <em>object or aim of an action</em>. It’s the thing we want to acheive through an action, or in most cases, a series of action.</p>
<p>You could say that a goal is an <em>internal representation</em> of a desired state. In contrast to a behavorist model of rewards and responses, goals serve a cognitive function for so-called “higher” or more sophisticated biological forms of life or creatures with a philosophy of mind. First, we imagine and visualize what we want and, in turn, take goal-directed actions that are in concordance with our plans for accomplishing it. We see this in various experiments on mammals (like certain monkeys) and cephalopods (like octopus) when confronted with a desired treat and a challenge to reach it. These animals, much like humans, are able to evaluate a situation, make a plan and execute. There might be some trial-and-error in reaching the solution but the goal intention is something they possess cognitively. They act on their goals.</p>
<p>For us, goals are more than just a one-off, single task target. Goals are hard and not easily accomplished. We commmonly think about goals as projects and endevours that involve multiple steps and take time. Unlike other human behavior, goals tend to require resources, skills, opportunities, teamwork and a good deal of figuring stuff out.</p>
<p>As such, psychologists make the following distinction:</p>
<ul>
<li><strong>Goal Intentions</strong>: These are the desired end state, or the something we have want to have but have yet to attain. A goal intention is often the traditional sense used when we refer to “goals.” The key question here is: what is the end state or goal we want to get?</li>
<li><strong>Goal Pursuit</strong>: Often referred to as goal striving, this is an action perspective of goals. As such goals as a goal pursuit take various actions. These goal-directed action encompass “all activities directed towards an “intended goal” (Achtziger, 2010). The key question here is: what are the goal actions necessary to pursue this goal intention?</li>
</ul>
<p>Goal psychologists have divided into two camps. The first camp focuses on the content of the goal and how this plays a key role in performance and eventual attainment. You can associate this camp with Latham and Locke’s specific challenging goals in <a href="http://www.markwk.com/science-of-goals.html">Goal Setting Theory</a>, Dweck’s learning goals and Higgin’s promotion goals.</p>
<p>The second camp focuses on how people are able to regulate the pursuit of their goals. This involves a deeper understanding of the self-regulatory agent (the goal do-er) in this pursuit and their various challenges, conflicts and decisions. For example, emotional control, goal selection, competiting goals, planning, mindsets and implementation intentions to name a few.</p>
<p>While setting goals does matter, goals often can only be obtained when our goal pursuit manages to complete a series of stages and is supported by various self-regulatory strategies.</p>
<p><strong>The term “goal” thus describes both goal intentions and goal pursuits. A goal intention is the objective we want to attain, while a goal pursuit is the multistage striving for that objective.</strong></p>
<h2 id="self-regulatory-view-of-goals">Self-Regulatory View of Goals</h2>
<p>It’s fun to set goals, but the problems of actual goal attainment are numerous and accute: not setting a goal, failure to get started, getting derailed, not taking goal action opportunities, overextending oneself, competing interests, poor pursuit, and many others (Gollwitzer & Sheeran, 2006).</p>
<p>The recognition that goal failure and goal performance depend on more than the content of our goal setting lies at the heart of the <strong>self-regulatory view of goals</strong>. It’s an active area of research and writing in the social sciences with many perspectives to consider.</p>
<p>According to the Theory of Self-Regulation, our goal behavior display a number of cognitive, behavioral and emotional processes over time and in different situations (ex. Karoly, 1993). Self-regulation has led psychologists to posit that our goal-action behavior can be divided into “discrete phenomena” (Gollwitzer, 1990) and to work towards a temporal, multistage conceptualization of the goal pursuit.</p>
<p>Heckhausen and Gollwitzer believe that goal pursuits exist as a series of stages and transitions that they and others call the Model of Action Phases (MAP) or Rubicon Model of Action Phases (Heckhausen & Gollwitzer, 1987; Heckhausen, 1991). They argue that in order to understand and to improve our goal pursuits, we first need to discern the constituent parts of goal striving.</p>
<h3 id="crossing-the-rubicon-from-predecisional-to-volitional-goal-actions">Crossing the Rubicon: From Predecisional to Volitional Goal Actions</h3>
<p>Before looking at its constitent phases, it is important to note that at the center of this model is a transition point, the point where we decide to take action towards a goal. We shift from thinking and deliberating about whether and what goals to planning and action.</p>
<p>Metaphorically, Heckhausen and Gollwitzer call this “crossing the Rubicon” in reference to Julius Cesaer’s overthrowing of Rome, and it refers to a recognizable shift in our mindset or psychological orientation.</p>
<p>Before we are in predecision phase of goal deliberation and ask ourselves questions like: What goals should I select? What goal is best and why?.</p>
<p>After crossing this transition point, we have made a decision and committed to our goal. Our thinking changes, and we now begin to ask questions like: How can I achieve this goal? What do I need to do to attain it?</p>
<h3 id="stages-boundaries-and-transition-points-of-the-goal-pursuit">Stages, Boundaries and Transition Points of the Goal Pursuit</h3>
<p>The (Rubicon) model of action phases postulates that there is a “course of action” or “series of consecutive phases” in our goal-directed behavior. This course goes through phases that have clear and recognizable boundaries. The different phases have different mindsets, tasks and underlying questions (Achtziger, Gollwitzer, 2010).</p>
<p>As anyone who pursues goals and projects can attest to, there is a difference between pondering and dreaming about goals and actual advancing on them. I think most people have experienced this difference when we go from the open mind wandering of wishes and dreams to the actualization of planning and doing. Psychologists refer to this as a shift from motivational, predecision phase to the volitional phase.</p>
<p>The model of action phases recognizes that a “transformation” occurs when a wish becomes a goal pursuit. Namely we move from “a fluid state of deliberating the value of a potential goal to a firm sense of commitment to its enactment, i.e. to the formation of a goal intention” (p. 277, Achtziger, 2010).</p>
<p>Here is a schematic of the four action phases and their boundaries:</p>
<p><img src="/generated/images/2019-resources/goals-model-of-action-phases-diagram-800-f1b183485.jpg" srcset="/generated/images/2019-resources/goals-model-of-action-phases-diagram-400-f1b183485.jpg 400w, /generated/images/2019-resources/goals-model-of-action-phases-diagram-600-f1b183485.jpg 600w, /generated/images/2019-resources/goals-model-of-action-phases-diagram-800-f1b183485.jpg 800w, /generated/images/2019-resources/goals-model-of-action-phases-diagram-1000-f1b183485.jpg 1000w" /></p>
<p>Source: “Chapter 11: Motivation and volition in the course of action” (A Achtziger; PM Gollwitzer, 2010).</p>
<p>The key transition points are between deliberation and goal decision, between planning and actual goal actions, and between goal actions and goal completion or evaluation.</p>
<h2 id="model-of-action-phases-overview">Model of Action Phases: Overview</h2>
<p>Broadly speaking, according to the model of action phases, the goal pursuit is divided into two parts:</p>
<ol>
<li>the motivational (what goal are we going to do? and how did our goal efforts go?) and</li>
<li>the volitional (what are my plans towards that goal? and what actions am I doing?).</li>
</ol>
<p>You might say at the center are two phases focused on planning what one should be doing and actually doing it. At the two ends of the model, we find processes associated with considerations about if the goal is the right one and how the recent goal activities went.</p>
<p>According to Achtziger and Gollwitzer, the Rubicon model addresses four questions:</p>
<ul>
<li>How do people select their goals?</li>
<li>How do they plan the execution of their goals?</li>
<li>How do they enact their plans?</li>
<li>How do they evaluate their efforts to accomplish a specific goal?</li>
</ul>
<p>Each of these questions is both a practical question facing the goal pursuer (e.g. what should I do with the goal I’m pursue at this moment in time?), and a theoretical and scientific question facing researchers (e.g. what is going on for goal pursuers and what are leads to success or failure at this juncture of their goal pursuit?).</p>
<h4 id="deliberation-what-goal-should-i-select">Deliberation: What goal should I select?</h4>
<blockquote>
<p>“Once subjects move from planning and goal-setting to the implementation of plans, they cross a metaphorical Rubicon. That is, their goals are typically protected and fostered by self-regulatory activity rather than reconsidered or changed, often even when challenged.”
Lyn Corno, The best laid plans, p. 15</p>
</blockquote>
<p>People have multiple wishes, desires, and dreams. These are things we might possibly want to do, but since there only so much time, energy, resources, etc., we have to decide on if we will pursue those goals or not.</p>
<p>This predecisional phase is where we weigh the desirablity and feasilbity of our goals: <em>Is the goal (or outcome of that goal) something I truly want and desire? And is that goal something that is feasible and I can reasonably expect to reach it?</em></p>
<p>Our mind-set here faciltates the <strong>core task of “choosing the most desirable wish that is also feasible”</strong> (p. 64, Gollwitzer, 1990). This leads to a cognitive tuning focused on information processing that is relevant to making that determination.</p>
<p>Experiments have shown that in this predecisional phase we are open-minded, more receptive to information in general, and show signs of critical thinking. We are attuned more to alternatives rather than the how-to and stategies of goal acheivement. We contemplate and have better recall of our open options rather than our plans.</p>
<p>There is on-going research on strategies you can use here, but one notable example that appears effective is <strong>mental contrasting</strong>, which involve reconciling the fantasy aspect of a goal with a realistic assessment of the obstacles (Oettingen, 2010).</p>
<h4 id="planning-what-should-i-do-and-when-and-where-should-i-do-it">Planning: What should I do? And when and where should I do it?</h4>
<blockquote>
<p>The task to be solved by the postdecisional (preactional) individual is planning when, where, and how to act in order to promote action initiation (p. 65, Gollwitzer, 1990)</p>
</blockquote>
<p>When we commit to a goal, which is sometimes called a “goal decision,” our psychological orientation or mindset changes, and we shift into <strong>volition</strong> or the process of making and acting on our goal decision.</p>
<p>This phase is about planning and strategizing with a focus on determing the required when, where and how so our goal pursuit can be successful.</p>
<p>Compared to the Deliberative Mindset, the <strong>Implementational Mindset</strong> displays more “closemindedness” in the sense that its “cognitive tuning” concentrates more on information relevant to task performance and ignores less-relevant information. We see individuals attuned to and devoting more time to planning.</p>
<p>In terms of their memory, they have less recall of alternative options, e.g. separate goals, and have easier recall of information related to planning, e.g. goal-relevant tasks. Basically they are less aware of so-called “incidental infomation” of other stuff and more in-tune with ways to move forward with their established goal.</p>
<p>There are two problems facing us after we’ve decided on a goal and before we have started acting on it:</p>
<ol>
<li>The first challenge is figuring out <strong>what actions we need to take</strong>. To some extent after committing to a goal, we know a few actions we might take but we likely haven’t determined all of the necessary steps nor determined the optimal best actions. So the planning stages involves some degree of learning and strategizing.</li>
<li>The second challenge is to <strong>determine the right opportunities for those goal-directed activities</strong>. Oftentimes we have to wait for the appropriate opportunity (when and/or where) to act on our goals. For example, the goal might require a certain amount of time, equipment, or time of day. A common reason why we fail at a goal is not seizing or recognizing these opportunities to act. Sometimes we have a goal but we haven’t planned the when and where for doing it.</li>
</ol>
<p>So, the questions facing us in this phase are two-fold: <strong>1. What actions do I need to take to move towards my goal?</strong> and <strong>2.) What situation, environment or context do I need to put in place so I take those actions?</strong></p>
<p>Basically this boils down to the challenge of implementation. To successfully reach a goal, we need to figure out the how, when and where.</p>
<p>The realization that implementation is key to goal realization has led to a heavy focus in the literature on this stage of action phases, both theoretically and practically. By understanding what goes on in implementation formation, we can better determine why we fail at goals. Also once we start to understand operationally what is happening, on the practical side, we can formulate and provide strategies to avoid those failures.</p>
<p>Self-regulation and implementation strategies are a key aspect to improving our goal pursuit, and there are quite a few areas and techniques that have been studied and proven effective, including the <a href="http://www.markwk.com/science-of-goals.html">type of goals we set</a>. A full dive into these strategies is outside the scope of this article, but one in particular is worth noting and that’s implementation intentions.</p>
<p><strong>Implementation intentions</strong> describes the goal stategy where you consider specific situational cues and formalate your behavior as a cued response in those situations. In contrast to pure goal intentions like “I want to acheive goal X”, implentation intentions are in a format like, “If situation Y arises I will initiate behavior Z.” We form a mental link between an act and a trigger. The point is to initate and automate the target activity based on a situational cue.</p>
<p>Often called “if-then” plans, implementation intentions have been well-researched, and there is robust support for the positive effect on goal attainment and task performance (Gollwitzer & Sheeran, 2006).</p>
<p>If you are looking to apply one lesson from the science of goals using action phases, implementation intentions is one of the best and most effective. Arguably it is something many of us already do when we put a meeting in a calendar or assign a block of time of the day to work on something. Both of these are situations where we’ve decided what we will do and assigned it to the when and where to do it.</p>
<h4 id="action-how-can-i-ensure-i-follow-through-on-my-actions-and-avoid-distractions">Action: How can I ensure I follow-through on my actions and avoid distractions?</h4>
<p>The Action Phase is one of the least discussed in the literature I’ve seen on model of action phases. This is partially due to the fact that the specific actions we take vary significantly from goal to goal.</p>
<p>The Action Phase is where we enact our strategies developed during the planning implementation phase. During the action phase it is crucial to stay on goal-relevant tasks and avoid anything that takes away you from those tasks. In order to reach a goal, we need to maximize goal-facilitating action and minimize anything that takes us away from those actions.</p>
<p>The <strong>Actional Mindset</strong> is associated with “steadfast pursuit” and deploys a few approaches when pursuing goals. The two most notable are:</p>
<ol>
<li>stepping up efforts when facing difficulties and</li>
<li>
<ol>
<li>resuming goal directed actions after interrupts.</li>
</ol>
</li>
</ol>
<p>When the going gets tough, one effective response is to work harder, and when we get off track, the best response is to resume activity as soon as possible.</p>
<p><strong>Strategically, the key focus should be to avoid “any halting of the flow of action that postpones goal achievement” (p. 66, Gollwitzer, 1990). Besides the actions themselves, the key actional task for sucessful goal directed activity is to shield ourself from distractions and tempations.</strong></p>
<p>There is an an overlap between the action phase in the model of action phases and the experience of flow (Csikszentmihalyi, 2008). “Flow” or “being in the zone” is the intense experience of being so involved in an activity that nothing else seems to matter. It’s a state of consciousness of total concentration and hypervigilance that has been witnessed in various athletes and creatives in peak performance (Kotler, 2014).</p>
<p>When it comes to pursuing our goals, flow provides a good way to measure how focused we are on actions and are able to avoid distractions, temptations and alternative goals. If we are in flow, then we are fully in the action phase, actively pursuing the task at hand.</p>
<h4 id="evaluation-did-i-reach-my-goal-and-how-did-i-do">Evaluation: Did I reach my goal and how did I do?</h4>
<blockquote>
<p>This mindset is “cognitive tuning towards information releveant to assessing the quality of the acheived outcome and the desirabliity of its consequences” (p. 281, Achtziger, 2010).</p>
</blockquote>
<p>According to the Model of Action Phases, goals go through a series a steps and might even be described as a cycle or feedback-loop.</p>
<p>We can’t plan or pursue a goal nonstop. Eventually it comes to an end or a point of reflection and evaluation. This post-actional phase recognizes that after completing a goal (successfully or not), we evalute it and ask questions. In particular, first we ask “whether the intended outcome has been achieved” and second, we wonder “whether the actual value of the goal striving matches its expected value” (p. 59, Gollwitzer, 1990).</p>
<p>We measure the results of our actions against the goal set and establish if the goal has been reached, and then we score the outcomes of our actions, their consequences, and their newly accessed value.</p>
<p>The associated mindset here is called the <strong>Evaluative Mindset</strong>. Like the other phases, the evaluative mindset carries with it its own psychological characteristics. This includes a more objective way of viewing and judging our personal situation and a return to more open-mindness where we explore alternatives.</p>
<p>Several questions typically come to mind:</p>
<ul>
<li>Did I acheive what I intended?</li>
<li>How well have I succeeded in acheiving my goal?</li>
<li>Can I consider this goal complete?</li>
<li>Did the action result in the positive consequences I anticipated?</li>
<li>Is the goal’s worth/value the same or different now?</li>
<li>If the goal failed or wasn’t obtained yet, should the goal be abandonned?</li>
<li>If I decide to continue, should I proceed with a different means or approach?</li>
</ul>
<p>In the evaluation phase, we look at our original goal set and determine if we hit the mark. In view of our recent efforts and our increased knowledge, we can again consider the desirability of that goal and reevaluate aspects of its feasibility.</p>
<p>Ultimately this will lead us to decide on our next step. This might be ending that goal pursuit, which is referred to as “goal deactivation,” or starting a renewed goal pursuit involving a revised goal setting, goal implementation and goal actions. In either case, our recent efforts have provided knowledge, experience and likely skills to improve our future goal performances.</p>
<h2 id="conclusion-recognize-the-stage-of-your-goals-and-tailor-accordingly">Conclusion: Recognize the Stage of Your Goals and Tailor Accordingly</h2>
<blockquote>
<p>When it is obvious that the goals cannot be reached,
don’t adjust the goals, adjust the action steps.</p>
<p>子曰: 射有似乎君子失诸正鹄反求诸其身 (中庸)</p>
<p>Confucius, quoted in <em>Book of Rites</em></p>
</blockquote>
<p>Goals are a complex behaviorial phenomon. While much of our scientific understanding about goal has improved, many mysteries remain. The temporal aspect of goals is a critical component since most of our goals take place over long time frames with deadlines. For example, during the goal pursuit, we receive feedback on our progress and modulate our specific behavior according to how much time is left and where are actions might contribute most effective to our goal success. This idea has been explored in attempts at computational modeling of the goal pursuit (e.g. Vancouver, 2010).</p>
<p>In this article, we focused on deepening our understanding of goals as a multiple stage pursuit. By reframing goals as a pursuit, it’s clear that goals are best understood and approached through these different stages. A typical goal pursuit goes through different stages, key transitions, and mindsets. The Rubicon Model of Action Phases shows that our goal striving involves a “consecutive series” of four stages: deliberation, planning, action and evaluation. Most important is the transformation from a predecision goal option to a committed goal intention once we decide on a goal.</p>
<p>For both a science of goals and for anyone pursuing goals, the Model of Action Phases provides a lot of useful and interesting ideas. For me, this model has contributed heavily to why I goals as a process or cycle, rather than just a target. It’s also made more conscious of clarifying at what stage a goal is in and applying effective goal management techniques for both goal review and goal implementation.</p>
<p>Goals are not just a representation of what we want to achieve, but an aspect of a more complex framework for goal-driven human action across time and across different situations. If we are going to reach our goals, we need recognize not only this distinction but find the appropriate strategies tailored towards our particular goal in that particular phase.</p>
<p>For example, when facing the challenge of deliberation, goal selection and goal setting, we can apply aspects of mental contrasting or goal setting theory. Similarly in the planning phase, it is most effective to use implementation intentions and if-then plans to prime us for the how, when and where of goal-orientated action. When it comes to goal actions, the key task is to maximize goal-directed activity and minimize distractions and temptations. This could include aspects of productivity and time management, like of “deep work” (Newport, 2016), or optimizing peak performance using “flow” (Csikszentmihalyi, 2009).</p>
<p>I believe that In order to optimize our goal pursuit, it can be helpful to clarify where our different goals fit with these different phases. This allows you can bring the best mindset (open-ness vs. close-minded focus) and ask the appropriate questions (deliberative- vs. implementation- vs. evaulative-oriented). And, of course, knowing what stage you are in with a particular goal can make you aware of some appropriate self-regulation and action control strategies.</p>
<p>In <a href="http://kops.uni-konstanz.de/bitstream/handle/123456789/1321/oettingen_gollwitzer_strategies.pdf?sequence=1">“Strategies of setting and implementing goals: Mental contrasting and implementation intentions”</a> (2010), one of my favorite acadmeic articles on the science of goals, Oettingen and Gollwitzer put it succinctly: <strong>the “psychology of goals suggests that successful goal pursuit hinges on solving two sequential tasks: goal setting and goal implementation” (p. 114).</strong> If you want to improve your goals, you need to understand and optimize your strategies for both. Their paper goes into depth about two of the best strategies mental contrasting and implementation intentions. But many other strategies and techniques remain to be researched and considered.</p>
<p>By understanding goal action phases, you start to recognize what stage your goals are in and apply optimal strategies. This act will not only help you better understand the science of goals but hopefully help you make the right decisions and actions towards acheiving them.</p>
<p><br /></p>
<p><br /></p>
<h5 id="sources">Sources:</h5>
<p>Achtziger, A., & Gollwitzer, P. (2010). Motivation and volition in the course of action. Retrieved from http://kops.uni-konstanz.de/bitstream/handle/123456789/1304/Achtziger_Gollwitzer_motivation_and_volition.pdf</p>
<p>Csikszentmihalyi, Mihaly. <em>Flow: The Psychology of Optimal Experience</em>. Harper Perennial Modern Classics, 2008.</p>
<p>Corno, L. (1993). The best-laid plans: Modern conceptions of volition and educational research. Educational researcher, 22(2), 14-22. Retrieved from http://www.david-dai.net/s/Corno1993.pdf</p>
<p>Gollwitzer, P. M. (1999). Implementation intentions: strong effects of simple plans. American psychologist, 54(7), 493. Retrieved from http://kops.uni-konstanz.de/bitstream/handle/123456789/10101/99Goll_ImpInt.pdf</p>
<p>Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta-analysis of effects and processes. Advances in experimental social psychology, 38, 69-119.</p>
<p>Heckhausen, Heinz, and Peter M Gollwitzer. “Thought Contents and Cognitive Functioning in Motivational Versus Volitional States of Mind.” Motivation and emotion 11.2 (1987): 101-20.</p>
<p>Heckhausen, J, and H Heckhausen. <em>Motivation and Action</em>. Springer (1991).</p>
<p>Karoly, Paul. Mechanisms of Self-Regulation: A Systems View. Annual review of psychology 44.1 (1993): 23-52.</p>
<p>Kotler, S. (2014). <em>The Rise of Superman</em>. Houghton Mifflin Harcourt.</p>
<p>Lewin, Kurt. “Vorsatz, Wille Und Bedürfnis.” Psychologische Forschung 7.1 (1926): 330-85.</p>
<p>Newport, C. <em>Deep Work: Rules for Focused Success in a Distracted World</em>. Hachette UK (2016)</p>
<p>Oettingen, G., & Gollwitzer, P. (2010). Strategies of setting and implementing goals: Mental contrasting and implementation intentions. Retrieved from http://kops.uni-konstanz.de/bitstream/handle/123456789/1321/oettingen_gollwitzer_strategies.pdf</p>
<p>Vancouver, J. B., Weinhardt, J. M., & Schmidt, A. M. (2010). A formal, computational theory of multiple-goal pursuit: Integrating goal-choice and goal-striving processes. <em>Journal of Applied Psychology</em>, <em>95</em>(6), 985.</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}When it comes to goals, we commonly conflate several different actions and phenomonon under this singular term of “goal.” But this one term hides an important range of dimensions and actions we undertake when we think about and endevour to reach our goals. One of the most crucial differences is between a goal as an intention and a goal as a pursuit.Tracking Your Writings and Note-Taking2019-02-28T00:00:00-08:002019-02-28T00:00:00-08:00http://www.markwk.com/track-writings<h2 id="how-to-track-analyze-and-understand-a-life-in-writing">How to Track, Analyze and Understand A Life in Writing</h2>
<blockquote>
<p><em>“For years I have looked for the perfect pencil. I have found very good ones but never the perfect one. And all the time it was not the pencils but me. A pencil that is alright some days is no good another day.”</em></p>
<p>John Steinbeck, “Journal of a Novel: The East of Eden Letters”</p>
</blockquote>
<p><br /></p>
<p><em>Can we track what we write? If so, how? And what can we use our writing tracking for?</em></p>
<p>As a self-tracker and an enthusiast of the data-driven life, <a href="http://www.markwk.com/category/track-everything">I track a lot of my life</a>, but as a writer, I find the tracking options rather limited. There are tools to log my writing time and how words I typed, but there is nothing that allows me to keep a complete history and data on all of my notes, drafts and final creations. I wanted a way to track what I write, not just my time or word count.</p>
<p>Fortunately, since I <a href="http://www.markwk.com/migrate-evernote-plaintext.html">migrated off Evernote</a>, I now write in <a href="http://www.markwk.com/plain-text-life.html">plain text files</a>. Plaintext files are a file format that is future-proof, flexible and portable. They are also trackable, and I am able to track my life in writing.</p>
<p>Using git, a popular way to manage software development, I have assembled a self-tracking method for keeping a complete history of my files, including each and every change I make, and for logging daily statistics on words added, files changed, and more. <strong>In short, with plain text files, git and a few automation scripts, we have a comprehensive and robust method of tracking our writings and notes.</strong></p>
<p>In this post, I want to share how to track your writings. By using plaintext files, git and a few scripts, I’ll show what it takes to record a complete history of your notes into git and also collect some high-level statistics of daily changes. Since it is important to know what data we are getting and the potential insights we can get from the data, I’ve also provided a starting point for some data analysis on that tracking data.</p>
<p>Hopefully, by the end of this post, you’ll have mastered the basics of managing and tracking your writings with plaintext files and git and equiped yourself with a way to comprehensive way to track your writings and notes in the future!</p>
<p><br /></p>
<p><em>NOTE 1</em>: You can find the code for this post at <a href="https://github.com/markwk/writing-tracker">Writing Tracker</a>.</p>
<p><em>NOTE 2</em>: For a more in-depth dive into writing and note-taking check out <a href="http://www.markwk.com/plain-text-life.html">The Plaintext Life: Note Taking, Writing and Life Organization Using Plain Text Files</a>, and for a step-by-step tutorial on migrating off of Evernote to plaintext files, see <a href="http://www.markwk.com/migrate-evernote-plaintext.html">Post-Evernote</a>.</p>
<!--more-->
<h3 id="proxy-metrics-writing-time-and-typing-word-count">Proxy Metrics: Writing Time and Typing Word Count</h3>
<p>I’ll admit that I’m a pretty obsessive tracker. I’ve even gone so far as to <a href="http://www.markwk.com/poop-tracker.html">track my poop</a> (don’t judge before reading!). When it comes to writing, historically I’ve used a number of metrics to track my writing.</p>
<p>For example, using RescueTime, I track my computer time and my time spent in apps. This allows me to know what percentage of my time is on writing-related tools. Similarly, using Toggl time, I manually turn on and off a time tracker. This allow me to log how long I actively spend on certain specific projects, especially client work but also my studies, writings and organizational chores. Basically, I know how much time I spend on writing on my computer.</p>
<p>According to my <a href="http://www.markwk.com/2019/01/year-in-data.html">2018 data analysis</a>, I spent 370 hours actively on writing, which was a 62% increase from the year before. This resulted in longer and better blog posts and considerable progress in my yet-to-be-published works. Here is a heat map comparing my writing time in 2017 vs. 2018:</p>
<p><img src="/generated/images/2019-resources/writing-time-2017-vs-2018-800-7929b34f8.png" srcset="/generated/images/2019-resources/writing-time-2017-vs-2018-400-7929b34f8.png 400w, /generated/images/2019-resources/writing-time-2017-vs-2018-600-7929b34f8.png 600w, /generated/images/2019-resources/writing-time-2017-vs-2018-800-7929b34f8.png 800w, /generated/images/2019-resources/writing-time-2017-vs-2018-819-7929b34f8.png 819w" /></p>
<p>One very noticable difference is that I have more days where I spent 4 hours or more writing in 2018. Most successful writers dedicate a period of 2-4 hours for their writing. I still find my writing routine somewhat inconsistent, but this is a step in the right direction.</p>
<p>While time logs are an interesting and important dataset for knowing if I’m sticking to my habit of writing, it is a <em>proxy metric</em>. It only measures the time I’m putting in. It doesn’t tell me much about my output, like how many words I type or what notes I worked on or the evolution of my corpus of notes and drafts.</p>
<p>When it comes to <em>my written output</em>, one way to track this is through how many words you type. For a couple years, I’ve recorded my creative typing number using a Mac app called <a href="">WordCounter</a>. This tool lets me track how many words I type per hour on certain designated programs (and exclude non-writing programs, like browser, email or a coding editor). This app doesn’t record what I type or in what specific document, but it does let me track how many “creative words” I type each day and using which applications.</p>
<p>Here are two examples:</p>
<p><img src="/generated/images/2019-resources/word-counter-examples-800-6d04c6ba2.jpg" srcset="/generated/images/2019-resources/word-counter-examples-400-6d04c6ba2.jpg 400w, /generated/images/2019-resources/word-counter-examples-600-6d04c6ba2.jpg 600w, /generated/images/2019-resources/word-counter-examples-800-6d04c6ba2.jpg 800w" /></p>
<p>On the left, we see the default stats provided in the menu by WordCounter, and, on the right, we see more advanced analytics provided by my <a href="https://github.com/markwk/alfred-workflow-wordcounterapp">Alfred Integration for WordCounter</a>. This workflow includes a way to export your daily data and a couple data visualization reports.</p>
<p>If you want to go even deeper, in my <a href="https://github.com/markwk/qs_ledger">QS Ledger</a> I’ve created a <a href="https://github.com/markwk/qs_ledger/tree/master/wordcounter">Data Analysis Jupyter Notebook on WordCounter</a>. This script will extract your data and provide a number of ways to visualize it. Here is the changes of my tool usage in 2018:</p>
<p><img src="http://www.markwk.com/images/2019-resources/wordcounter-writing-analysis-3.png" alt="Writing App Usage in 2018" /></p>
<p>This chart provides shows which months I was most prolific (May, Nov, and Dec) and the change from using almost exclusively Evernote and TextWrangler to my <a href="http://www.markwk.com/plain-text-life.html">current tool stack of The Archive and Typora</a>.</p>
<p>Overall, WordCounter is a great way to track my writing activity. It tells me how many creative words I write per hour and which programs. This allowed me in my <a href="http://www.markwk.com/2019/01/year-in-data.html">year-in-review</a> to see which , which days of the week I write the most (Monday and Wednesday) and even which hour of the day (10am and 11am).</p>
<p>While time logs and word count output are a pretty good start to tracking how much I write, it leaves a lot to be desired. Both are proxy metrics to what I’m actually trying to track and understand, namely my writing output. I could use a similar proxy metric like my Todoist log of writing tasks completed too.</p>
<p>Basically, time logs are a proxy to the eventual output since it tells me about the time I’m putting in, and my word count tracker is a proxy to what I eventually create as blog posts and book chapters. But neither really helps me understand the evolution and genesis of my actual writings. Neither provide as much data as should be possible if I want to have something like a life log of my complete writing life too.</p>
<p>In order to understand much more on my writings, we are going to need to start tracking my actual notes and writings!</p>
<h3 id="want-to-track-your-writings-write-and-store-your-notes-in-plaintext-files">Want to Track Your Writings? Write and Store Your Notes in Plaintext Files</h3>
<p>There are a lot of tools out there for writing and note-taking. Most are full-featured apps that handle both the role of editing and organizing. These note apps can be pretty useful and convenient, but they come with a few limitations.</p>
<p>The biggest limitation is where your actual text gets stored. In nearly all programs, the notes you write are stored or locked inside the program and inputed into some propertary text format. This limits your ability to port your data other systems. Also, having the input in a non-standard format makes it hard to use it differnet mediums. For tracking purposes, this “locked in” aspect makes it almost impossible to get any data on your writings too.</p>
<p>In order to track our writings we are going to either need a program that does the tracking for us or get our writings in a file format we can track ourselves. As far as I know, none of the major note-taking applications, like Evernote, OneNote, or Keep provide accessible tracking data on your usage nor an easy way to track this yourself. So, for tracking purposes we are going to need access to file format that exists outside of the tool, and for us that means using plaintext files.</p>
<p>Plaintext files are one of the oldest file formats in computing and are the basis of most coding. The chief advantages of plaintext files is that you can open them anywhere, and they’re portable, future-proof, and trackable.</p>
<p>Basically, keeping your notes in plaintext files will never require a subscription, lock away features, or go out of business. It’s free and here forever. And you can choose from any number of tools to edit and manage your files.</p>
<p>So, if you want to track your actual writings and notes (and not just your writing time or global typing, word count), the best way to start doing this is to write and store all of your notes in plaintext files.</p>
<p>Now when it comes to the actual tracking, the best approach in my opinion is git. Git or more specifically git version control is the most popular method for tracking and managing code today. Much like how one can track code and software development, with plaintext files I can now track my writing and writing process too.</p>
<p>Let’s now to turn to setting up our writing tracker with git.</p>
<h3 id="what-is-git-coders-friend-for-tracking-your-files">What is Git? Coder’s Friend for Tracking Your Files</h3>
<p>In order to better understand and better track my writings, I needed to start tracking my actual notes, not just my time or my typing. Unfortunately, none of the major note-taking applications provide much tracking data nor make it easy to track your files. This is why for trackability it was necessary to keep everything in files. This make it possible to track changes using git.</p>
<!--About Git: [[201902221417]]-->
<p>Git is a distributed system for tracking changes in files. It was created by Linus Torvalds in 2005 to help manage the development of the Linux, an open-source operating system. It was designed for and is typically used by programmers to manage changes in code, but it can be used to track changes in any set of files.</p>
<p>Git works a bit like a filesystem in the sense that you store the latest version of the files, along with a log of changes over time. These changes over time are called diff’s and are the differences between two versions or commits in time. Using git, multiple people can work on the same pieces of code and later merge those changes together.</p>
<p>An intro to git goes beyond this post. There are a lot of great resources online for learning and installing git. Two I recommend are the <a href="http://rogerdudler.github.io/git-guide/">Git Guide</a> and <a href="https://try.github.io/">Github’s Try Git</a>. Installing and learning basic commands for Git should only take 30 minutes to an hour. Most git-related challenges you encounter can be solved through Google Search, StackOverflow, and try and error.</p>
<p>Once you have git installed, you are now able to track the changes to a collection of files overtime. Typically developers make manual “commits” as they work on the code. Basically once you’ve finished your work you save your changes into the git repository with a short log message. For example, “Fix for bug 123” or “Initial Work on Feature XYZ.” The commit history is a central feature of git since it provides a complete log of any and all code changes over time.</p>
<p>When it comes to tracking our writings, we can also use git. All you need is a directory of files. For example, each time you make changes, you can save and log them into the repository. This is how some writers use git.</p>
<p>But if we want to go beyond this manual process we are going to need to automate our notes tracking with git. This is why I created <a href="https://github.com/markwk/writing-tracker">Writing Tracker</a>, a git-based notes tracker and statistics counter.</p>
<h3 id="how-to-automatically-track-plaintext-notes-with-git">How to Automatically Track Plaintext Notes with Git</h3>
<p><em>NOTE: You can find the bash script code for this section at https://github.com/markwk/writing-tracker.</em></p>
<p><a href="https://github.com/markwk/writing-tracker">Writing Tracker</a> is a collection of scripts for tracking and data analysis of writings or plaintext notes. My main goal and the goal of the project is to provide a simple tracking setup for any directory of plaintext files. In turn, I’ve included both a simple data report and a jupyter notebook for more advanced data analysis.</p>
<p>Writing Tracker has three main components:</p>
<ul>
<li>writing-tracker.sh: script to track daily changes in a directory of plaintext files</li>
<li>report.py: simple python script to graphically plot latest stats with pandas and matplot</li>
<li>writing_data_analysis.ipynb: Jupyter notebook in Python to analysis of writing stats.</li>
</ul>
<p>In order to get started check out the installation and setup steps](https://github.com/markwk/writing-tracker#installation-setup-and-usage-writing-tracker-with-bash-and-git).</p>
<p>The core of the project is writing-tracker.sh. This is a bash script. A bash or shell script is a way to run a series of command line or terminal commands.</p>
<p>Basically, I’ve created an automated process to look first at the changes to your writing files directory and commit them to a git repo. The code then runs a series of calculations (new files, word count, etc) to see what changed. These stats are stored to a separate CSV spreadsheet file. Finally, the files changes are committed with a basic log message of the date and a few key statistics.</p>
<p>In the end, we’ve stored a complete history of these files and we’ve calculated a daily breakdown of noteable changes.</p>
<p>Let’s walk through what the script does step-by-step.</p>
<p>At the top of the script are a number of variables you configure for where your directory is and where to store your output statistics.</p>
<p>The first real step in the script starts with git:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Navigate to target directory and start git staging</span>
<span class="nb">cd</span> <span class="s2">"</span><span class="nv">$TARGET_DIR</span><span class="s2">"</span>
git add <span class="nb">.</span>
</code></pre></div></div>
<p>This adds all the files to the repo so we can parse the recent changes. We then run <code class="language-plaintext highlighter-rouge">git status</code> and use grep to extract and count the various file changes:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># File Counts</span>
<span class="nv">total_files</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">ls</span> <span class="nt">-1q</span> <span class="k">*</span> | <span class="nb">wc</span> <span class="nt">-l</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s1">'[:space:]'</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">files_changed</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>git status | <span class="nb">wc</span> <span class="nt">-l</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s1">'[:space:]'</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">files_added</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>git status | <span class="nb">grep</span> <span class="s1">'new file'</span> | <span class="nb">wc</span> <span class="nt">-l</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s1">'[:space:]'</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">files_modified</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>git status | <span class="nb">grep</span> <span class="s1">'modified'</span> | <span class="nb">wc</span> <span class="nt">-l</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s1">'[:space:]'</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">files_deleted</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>git status | <span class="nb">grep</span> <span class="s1">'deleted'</span> | <span class="nb">wc</span> <span class="nt">-l</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s1">'[:space:]'</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">files_renamed</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>git status | <span class="nb">grep</span> <span class="s1">'renamed'</span> | <span class="nb">wc</span> <span class="nt">-l</span> | <span class="nb">tr</span> <span class="nt">-d</span> <span class="s1">'[:space:]'</span><span class="si">)</span><span class="s2">"</span>
<span class="nb">echo</span> <span class="s2">"Files: total files: </span><span class="nv">$total_files</span><span class="s2">, total changed: </span><span class="nv">$files_changed</span><span class="s2">, added "</span> <span class="nv">$files_added</span>, <span class="s2">"modified "</span> <span class="nv">$files_modified</span>, <span class="s2">"deleted "</span> <span class="nv">$files_deleted</span>, <span class="s2">"renamed"</span> <span class="nv">$files_renamed</span>
</code></pre></div></div>
<p>Similarly we use <code class="language-plaintext highlighter-rouge">git diff</code> to parse and calculate word counts in those differences:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Word Counts | Credit: https://gist.github.com/MilesCranmer/5c7d86c8740219355d2dfdb184910711</span>
<span class="nv">words_added</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> <span class="nt">--word-diff</span><span class="o">=</span>porcelain |grep <span class="nt">-e</span><span class="s2">"^+[^+]"</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nv">words_deleted</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> <span class="nt">--word-diff</span><span class="o">=</span>porcelain |grep <span class="nt">-e</span><span class="s2">"^-[^-]"</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nv">words_duplicated</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> |grep <span class="nt">-e</span><span class="s2">"^+[^+]"</span> <span class="nt">-e</span><span class="s2">"^-[^-]"</span>|sed <span class="nt">-e</span><span class="s1">'s/.//'</span>|sort|uniq <span class="nt">-d</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nb">echo</span> <span class="s2">"Words: added "</span> <span class="nv">$words_added</span>, <span class="s2">"deleted "</span> <span class="nv">$words_deleted</span>, <span class="s2">"duplicated"</span> <span class="nv">$words_duplicated</span>
</code></pre></div></div>
<p>Since I use tags and special reference keys, I do some specific checks to count those too.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># hashtag counts (note: excluding hashed tagged words like #1234)</span>
<span class="nv">hashtags_added</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> <span class="nt">--word-diff</span><span class="o">=</span>porcelain | <span class="nb">grep</span> <span class="nt">-e</span><span class="s2">"^+"</span> | <span class="nb">grep</span> <span class="nt">-o</span> <span class="s1">'#[a-zA-Z]\+'</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nv">hashtags_deleted</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> <span class="nt">--word-diff</span><span class="o">=</span>porcelain | <span class="nb">grep</span> <span class="nt">-e</span><span class="s2">"^-"</span> | <span class="nb">grep</span> <span class="nt">-o</span> <span class="s1">'#[a-zA-Z]\+'</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nb">echo</span> <span class="s2">"Hashtags: added "</span> <span class="nv">$hashtags_added</span>, <span class="s2">"deleted "</span> <span class="nv">$hashtags_deleted</span>
<span class="c"># Count of citations references in special format like #123.</span>
<span class="c"># Used in the case of a reference manager like Bookends or Zotero.</span>
<span class="nv">refs_added</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> <span class="nt">--word-diff</span><span class="o">=</span>porcelain | <span class="nb">grep</span> <span class="nt">-e</span><span class="s2">"^+"</span> | <span class="nb">grep</span> <span class="nt">-o</span> <span class="s1">'#[0-9_]\+'</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nv">refs_deleted</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--cached</span> <span class="nt">--word-diff</span><span class="o">=</span>porcelain | <span class="nb">grep</span> <span class="nt">-e</span><span class="s2">"^-"</span> | <span class="nb">grep</span> <span class="nt">-o</span> <span class="s1">'#[0-9_]\+'</span>|wc <span class="nt">-w</span>|xargs<span class="si">)</span>
<span class="nb">echo</span> <span class="s2">"References: added "</span> <span class="nv">$refs_added</span>, <span class="s2">"deleted "</span> <span class="nv">$refs_deleted</span>
</code></pre></div></div>
<p>The final result is that I’ve recorded the daily changes overall and stored a few data points on what changed. For example, I’ve got data on how many files were added, changed, deleted, etc., and I know how many words were added or removed. Additionally, I’ve created some special commands to get more relevant data like number of tags added.</p>
<p>Once this is all completed, we run two more commands:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Save stats as new line with date to local csv</span>
<span class="nb">echo</span> <span class="k">${</span><span class="nv">YESTERDAY</span><span class="k">}</span>, <span class="k">${</span><span class="nv">CURRENTDATETIME</span><span class="k">}</span>, <span class="nv">$total_files</span>, <span class="nv">$files_changed</span>, <span class="nv">$files_added</span>, <span class="nv">$files_modified</span>, <span class="nv">$files_deleted</span>, <span class="nv">$files_renamed</span>, <span class="nv">$words_added</span>, <span class="nv">$words_deleted</span>, <span class="nv">$words_duplicated</span>, <span class="nv">$hashtags_added</span>, <span class="nv">$hashtags_added</span>, <span class="nv">$hashtags_deleted</span>, <span class="nv">$refs_added</span>, <span class="nv">$refs_deleted</span> <span class="o">>></span> <span class="nv">$DATA_FILE</span>
<span class="c"># Commit Changes to Git with Custom Message</span>
<span class="nv">commit_msg</span><span class="o">=(</span><span class="s2">"</span><span class="nv">$YESTERDAY</span><span class="s2"> Daily Writing Stats: Words Added: </span><span class="nv">$words_added</span><span class="s2">, Files Added: </span><span class="nv">$files_added</span><span class="s2">"</span><span class="o">)</span>
<span class="nb">echo</span> <span class="nv">$commit_msg</span>
</code></pre></div></div>
<p>This puts the saved stats into a CSV file and commits all of the file changes into the repo with a short message.</p>
<p>Using cron, which I high recommend to anyone intending to use this, you can automate this script to run once a day, like midnight or at 1am. Techinically I could tweak it to run every hour, but once a day is fine for my purposes.</p>
<p>When it comes to tracking my writings, I think plain text files and git make for a robust and powerful approach. It also gives us a ton of data too. All I needed to do was automate it with bash and export the stats to a CSV.</p>
<h3 id="inital-data-analysis-and-data-visualization-of-my-writings-and-note-taking">Inital Data Analysis and Data Visualization of My Writings and Note-Taking</h3>
<p><em>NOTE: You can find a Python Script and Jupyter Notebook for this section at https://github.com/markwk/writing-tracker.</em></p>
<p>Now that we’ve got our git writing tracker working and let it run for a few weeks or months, we should have some data to start looking at.</p>
<p>There are two ways we could do this data analysis: a git-based analysis or daily stats approach.</p>
<p>Since we have a complete history in git, we could use some git stats and git repository analysis tools. For example, <a href="https://github.com/IonicaBizau/git-stats">Git Stats</a> provides some cool visualisations for local git statistics, and <a href="https://github.com/arzzen/git-quick-stats">Git Quick Stats</a> gives a view into your commit history. Neither of these quite fit our needs but they do give an idea of what might be possible.</p>
<p>This git analysis approach would allow us to get a raw and complete look at the evolution of our notes respository. We could see specific changes in any file over time. This is definitely the kind of deep analysis I intend to do one day, especially towards my long-term objective to study the geneology of my creative ideas. For now, it’s a bit beyond our needs or what I want to look at in this post, but having this kind history is exactly why we are tracking with git.</p>
<p>Instead, we are going focus on using our daily stats file. This is a bit more high-level but it still gives us a new perspective on how and where you write.</p>
<p>As we looked at in the last section, each day our writing tracker script logs some data points in a CSV spreadsheet along with a git commit. This gives us a bunch of daily statistics we can use to understand our writings. This stats file can be used in a simple spreadsheet or a more complex data visualization tools like Google Data Studio or Python.</p>
<p>As starting point, I’ve created a file called <code class="language-plaintext highlighter-rouge">report.py</code> which allows for a quick bar chart visualization of my writings. Looking at the last 7 days, let’s compare three areas: my creative words typed (in blue) and my actual writing output of my notes and drafts (in green) and my project notes (in purple).</p>
<p><img src="/generated/images/2019-resources/comparison-of-writing-words-tracking-800-2acbb67cb.png" srcset="/generated/images/2019-resources/comparison-of-writing-words-tracking-400-2acbb67cb.png 400w, /generated/images/2019-resources/comparison-of-writing-words-tracking-600-2acbb67cb.png 600w, /generated/images/2019-resources/comparison-of-writing-words-tracking-800-2acbb67cb.png 800w, /generated/images/2019-resources/comparison-of-writing-words-tracking-1000-2acbb67cb.png 1000w" /></p>
<p>These reports quickly show there is a discrepancy between my total typing count and projects vs. writing notes. For example, in this sample data, I have a day where I mostly work on my projects and did no actual creative writings. So, while I did type, I didn’t type much in the way of creative writings output. It was all project-related. This was a gap I couldn’t see with just my word typing data.</p>
<p>In <a href="https://github.com/markwk/writing-tracker/blob/master/writing-data-analysis/writing_data_analysis.ipynb">writing_data_analysis.ipynb</a>, I’ve provided a starting point to a more expansive data analysis. I currently trackthree directories of files: my writings/notes, my coding notes and my project, notes., and the notebook examines changes in my writing output behavior over time, in particular the difference between file changes and words added in those three areas.</p>
<p>Here are two breakdowns:</p>
<p><img src="/generated/images/2019-resources/writing-tracking-data-analysis-example2-800-232768dd9.png" srcset="/generated/images/2019-resources/writing-tracking-data-analysis-example2-400-232768dd9.png 400w, /generated/images/2019-resources/writing-tracking-data-analysis-example2-600-232768dd9.png 600w, /generated/images/2019-resources/writing-tracking-data-analysis-example2-800-232768dd9.png 800w, /generated/images/2019-resources/writing-tracking-data-analysis-example2-961-232768dd9.png 961w" /></p>
<p><img src="/generated/images/2019-resources/writing-tracking-data-analysis-example1-800-5ea46c0e6.png" srcset="/generated/images/2019-resources/writing-tracking-data-analysis-example1-400-5ea46c0e6.png 400w, /generated/images/2019-resources/writing-tracking-data-analysis-example1-600-5ea46c0e6.png 600w, /generated/images/2019-resources/writing-tracking-data-analysis-example1-800-5ea46c0e6.png 800w, /generated/images/2019-resources/writing-tracking-data-analysis-example1-958-5ea46c0e6.png 958w" /></p>
<p>These visualizations quickly highlight where my writing was focused more on project and where it was creative output. I can also see which days too. These patterns might be useful for seeing a day of the week effect or explore correlations with other aspects of my life, like sleep, exercise, and <a href="http://www.markwk.com/hrv-for-beginners.html">HRV</a>.</p>
<p>Obviously these are pretty early results using my writing tracker, but I think it shows a lot of potential going forward as I attempt to peer deeper into the creative process both intellectually and with data.</p>
<p>The main thing to remember is that since all of these files and their changes are logged daily into a git we have a complete historical record of our note-taking and drafts. Everything has been tracked and stored. It thus becomes possible to minutely examine how an individual note, idea or draft evolves. I’ll be able to see how a cluster of notes on a certain topic emerges, becomes a draft and maybe eventually transforms into something larger, like a blog post, article or book.</p>
<p>Obviously it’s still early, but we have this power and potential since we are now tracking our writings using plain text files and git.</p>
<h2 id="conclusion-steinbeck-cared-about-his-pencils-i-care-about-tracking">Conclusion: Steinbeck Cared About His Pencils, I Care About Tracking</h2>
<p>As John Steinbeck was finishing one of his last masterpieces, <em>East of Eden</em>, he was still pondering his writing desk, his creative process, and even his preferred pencils (and their sharpness!). Steinbeck was one of the most well-known writers of the 20th centry and produced some of the most iconic books in American literature, and yet there he was thinking about his tools and tinkering with his workspace.</p>
<p>In <em>Journal of a Novel: The East of Eden Letters</em> (1990), Steinbeck describes his creative journey, mental state and his wandering mind. He pursued a family life. He had hobbies like woodworking and an active interest in travel. In his late 40s at the time of writing that novel, he also still found time to “invent a paperweight for an inclined desk” and a “tool rack.” These are two of several instances where he discussed the physical tools and surfaces he worked on. In the often emphemeral realm of creativity, his journal entries remind us of how physical writing is, especially for pre-computer writers. It also show us how even successful writers and creatives still procrastinate and still think a lot about the tools they use to create.</p>
<p>Ironically, we find this procrastination/tool obsession in quite a few famous writers. It’s common for creatives to think about and even blame the tools we use, especially when we get stuck. It’s a form of procrastination but also a symptom of seeking improvements in the things we can control. <strong>Steinbeck cared about his pencils. I care about tracking.</strong></p>
<p>In this post, we took a look at how to track your writings and notes with plaintext files, git and an automation script. We also briefly explored the data we collected and how to visualize and how it. While there might be other metrics and ways to track your writing, I’d argue that there probably isn’t anything more comprehensive and robust as tracking your writing history with git.</p>
<p>The end result of a git-based writing tracking method is that I’m now able to keep a complete historical log of everything I write, including drafts, smart notes, project notes and more. I now know how many words I added (or deleted) in a day on specific documents and any changes I make on an individual draft or note. Using a spreadsheet or a more advanced data analysis tool, I can visualize the evolution of a day in writing, a piece of writing, certain groupings of notes and much, much more. There is even the inherent potential here to provide a geneology of certain idea and whatever you end up creating from those initial fragments.</p>
<p>As a writer and software engineer myself, I often think about my tools, especially tracking tools. In lieu of actual writing, I can even can even become a bit too engrossed in my tinkering with my tools, management processes, and self-tracking. Fortunately I can take some degree of solace in writers like Steinbeck who also procrastinated, thought about his tools and even blamed his pencils!</p>
<p>As he puts it in a moment of writerly self-awareness:</p>
<p><em>“For years I have looked for the perfect pencil. I have found very good ones but never the perfect one. And all the time it was not the pencils but me. A pencil that is alright some days is no good another day.”</em></p>
<p>Creativity demands much of our minds but, it doesn’t require much in the way of tools, especially in writing. Steinbeck wrote mostly with pencil. The reality is that the tools are just a medium of creation, and most of us can get by on the basics of a simple computer or even a notebook and a pen. We shouldn’t blame the tools nor go on an endless quest from tool to tool. <strong>If you adhere to the concept of artist as craftsman, what matters is that you show up and do the work.</strong></p>
<p>At the same time, assuming you are indeed doing the work, there is also nothing wrong with spending a bit of time on your tools and even a bit of “inventing,” as Steinbeck did. I think this is a case of needing <strong>space for both the work and the work about the work</strong>, which you might meta-cognition or meta-doing about the work.</p>
<p>Writer and professor Cal Newport embodies these two roles well. Newport is famous for his concept of “deep work,” which is the idea of removing distractions and finding focused periods of intense engagement. Newport has published both academic papers and mainstream books and exemplifies diligent output. At the same time, in his blog posts, I find elements of meta-cognition and the tinker. For example, in <a href="http://calnewport.com/blog/2008/11/11/plantxt-the-most-effective-productivity-tool-that-youve-never-heard-of/">Plan.txt</a> Newport recounts a simple way of managing his day-to-day, and, in <a href="http://calnewport.com/blog/2007/11/08/are-you-effective-or-just-busy-calculate-your-churn-rate-to-find-out/">Are you effective or just busy?</a>, he challenges concepts of productivity and proposes a method called churn rate. Both posts show him thinking about how he works and “inventing” processes to make his work more effective, better organized and tracked.</p>
<p>I believe that when faced with being creative and innovative, tools we like and processes that fit how we like to work do matter. The right tools help us stay organized and in flow. I’d even argue that the <a href="http://www.markwk.com/smart-notes.html">right tools, habits and systems matter immensely in pursuit of organized, creative mind</a>. Tinkering with your tools (or in my case tracking tools) also give us a beneficial way to “procastinate.”</p>
<p>Like Steinbeck, I shouldn’t blame my tools. As he put it, “it was not the pencils [to blame] but me.” All the same, even though we know it’s impossible to find it and eventually we must accept whatever works, it’s ok seek out the “perfect pencil” or whatever is your digital equivalent.</p>
<p><br /></p>
<p>Best of luck and happy tracking!</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}How to Track, Analyze and Understand A Life in WritingThe Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files2019-02-14T00:00:00-08:002019-02-14T00:00:00-08:00http://www.markwk.com/plain-text-life<blockquote>
<p>“The power of the unaided mind is highly overrated. Without external aids, deep, sustained reasoning is difficult.”</p>
<p>Don Norman, professor and author of <em>The Design of Everyday Things</em></p>
</blockquote>
<p>External aids, especially writing, are the key to sustained learning and a creative life. The goal of your productivity, writing or note-taking systems should be to enable you to think clearly, stay organized, learn, and create. They should augment your ability to reason, to develop connections across knowledge, and produce a targeted output.</p>
<p>There are a lot of tools that can help you in this pursuit. We live in a world of nearly endless options for productivity and writing software. Personally, I’ve tried many. But sometimes the best solution is one of the simplest and oldest. For me, that solution was “downgrading” to plain text files as my primary means for note-taking, writing, knowledge management and life organization.</p>
<p>Rather than a fully featured notes or writing tool, I now have a bunch of plain text files and a lot of them. The files themselves are simple, can be edited on any system, and are future-proof. I write in <a href="http://daringfireball.net/projects/markdown/">markdown</a>. I use plain text files not only for my writings, study notes and note-taking but also for my goals, organizational, project notes.</p>
<p>I call this method and approach a <strong>plain text life</strong>, by which I mean <strong>a note-taking, organizational, and writing system based on plain text files</strong>.</p>
<p>The files and notes themselves have been intentionally designed to be “organized” as network of information. Practically-speaking this translates into notes stored in a few directories, tagged, and connected together using a links. The end result is a loosely-coupled web of notes. It’s evolving, has emergent properties and is trackable too.</p>
<p><strong>This setup helps me focus on what matters: writing and keeping my projects, ideas and thinking organized and interconnected.</strong></p>
<p>In this post, I want to share my take on the “plain text life” and how using plain text files and combination of tools, best practices, and organizational principles can unlock a powerful and efficient framework for writing, thinking, note-taking, project management, goal tracking, or whatever you are working on.</p>
<p>This post is divided into four sections with four questions:</p>
<ul>
<li>Why plain text files and what are the current limitations?</li>
<li>What tools can I use for writing and managing plain text files?</li>
<li>How to stay organized?</li>
<li>What are my notes for and how to organize towards my creative, learning or organizational goals?</li>
</ul>
<p>At its core the plain text life is just a just a bunch of text files, but hopefully you too can assemble a powerful framework for staying organized, writing, learning and creating.</p>
<p>Let’s get started exploring a plain text life!</p>
<p><br /></p>
<p><strong>NOTE</strong>: For a how-to post on migrating off of Evernote, check out <a href="http://www.markwk.com/migrate-evernote-plaintext.html">Post-Evernote: How to Migrate Your Notes, Images and Tag into Plain Text Markdown</a>.</p>
<!--more-->
<h3 id="the-why-advantages-and-disadvantages-of-plaintext-files">The Why: Advantages and Disadvantages of Plaintext Files</h3>
<blockquote>
<p>“Writing is thinking. To write well is to think clearly. That’s why it’s so hard.”</p>
<p>David McCullough</p>
</blockquote>
<p>Writing is arguably <em>the</em> critical ingredient to how we think and learn. If you can’t write about something coherently and intelligibly, then your thinking on that topic or subject is vague and incomplete.</p>
<p>Similarly, I’d argue writing is a key aspect to personal and professional organization too. Often through lists, note-taking, project management tools, or a process journal, we write out our plans, goals, intentions and other aspects that clarify what we want to accomplish. Writing allows us to express vague feelings and turn them into intentions and goals.</p>
<p>Writing is also a way to keep track of things. It is argued that written language evolved largely as a mean to help us keep track of early business transactions. Writing is central factor in externalizing our memory. It also provides how we pass knowledge from generation to generation.</p>
<p>Over the years, I’ve used a number of tools to help me write, learn, and stay organized. I recently <a href="http://www.markwk.com/migrate-evernote-plaintext.html">migrated off of Evernote</a>. I elected not to switch to another note-taking app and instead started using just plain text files.</p>
<h5 id="key-advantages-of-plain-text-files">Key Advantages of Plain Text Files:</h5>
<p>Plain text will never require a subscription, lock away features, or go out of business. It’s free and here forever.</p>
<ul>
<li><strong>Flexibility: Open Them Anywhere</strong>: Plain text files are the most flexible file format we have. They can be opened by hundreds, if not thousands, of applications. Since they are a basic component of computers, you can even open them on the computer command line.</li>
<li><strong>No New Tools</strong>: We all get excited by new stuff, and for productivity junkies, we perk up with excitement to try out a new piece of software. Unfortunately tools come and go, and switching system can be wasteful and counterproductive. Switching to plain text means you never need to migrate to a new tool.</li>
<li><strong>Portability</strong>: By portability I mean that your files can be moved to and from different operating systems, platforms and devices and you can still open them. Whether you are on a Mac, Linux, Windows or some future tool, you’ll be able to open and edit your plain text files there.</li>
<li><strong>Future-Proof</strong>: A while back I discovered several writings I did in high school using Windows 95 and a version of Microsoft Word. While I was able to eventually open the files, the formatting had been lost. This revealed a potential danger in locking your words into a format that may not be around forever. Fortunately, while other file formats may come and go, plain text files will remain.</li>
<li><strong>Trackable</strong>: While self-tracking may not be a priority for everyone, if you are someone like me who <a href="http://www.markwk.com/category/track-everything/">likes to track</a>, plain text files provide the most trackable file format. As I’ll share in detail in a separate post, plain text files allows you to leverage Git, a common code management system, to track your notes and writings. This lets you see your daily changes in topics and word counts as well as the evolution of your notes, projects and manuscripts. You can see a <a href="https://gist.github.com/markwk/c85a8a72bc8c03d0f510262bb5219a34">git-based notes tracking script example</a> here.</li>
</ul>
<h5 id="notable-disadvantages-of-plain-text-files">Notable Disadvantages of Plain Text Files:</h5>
<p>There are also notable concerns with using plain text files. Chief among them is the requirement for more personal discipline and personal organization. By not using someone else’s designed setup, you have to make your own choices on how to stay organized.</p>
<ul>
<li><strong>Less Unified, Weaker User Experience:</strong> Unlike a complete note-taking or writing tool like Evernote, Ulysses, or Simple Note, editing and managing plain text files can feel more fragmented. Instead of a single app, you just have a bunch of files. This can make it feel less unified and provide a weaker user experience. Several tools, like The Archive and nvALT, provide organizational solutions on-top of plain text files. These include file and tag search. This brings much of the same feel and user experience as a complete note-taking tool, while keeping all of your files in plain text.</li>
<li><strong>Poorly Designed Tools for Writers and Note-Takers</strong>: Historically plain text editors were developed for software engineers and coders. Most are (or were) free or open source. This focus on writing code and being open source has resulted in many plain text editors that are not very attractive or well-designed for writers. They can be a bit intimidating to use too. Fortunately, with the rise of markdown many new and beautiful editors have appeared.</li>
<li><strong>Search and Discoverability Challenges:</strong> For me the key disadvantage since switching to plain text files has been search and discoverability. Since its indexing and search functionality was so strong, Evernote Search allowed you to get away with loose or poor note organization. You didn’t need to be that organized since search could still find what you were looking for. Plain text files can also be searched but the results are not as strong as other tools. Personally, I’ve realized that search may not be the best solution for dealing with a complex and growing network of notes and writings. As such search is not as important for me.</li>
<li><strong>Requires More Personal Discipline and Organizational Habits</strong>: Like a junk drawer or an unsorted inbox, things can get get pretty disorganized using plain text files. By default plain text files setup does force any kind of structure. You’ll need to develop and follow your own process.</li>
</ul>
<p>While it might seem like there are a lot of disadvantages or at least challenges to a plain text file-based setup, they might all be boiled down a singular question: <em>How to stay organized with plain text files?</em></p>
<p>Let’s at check out a few plain text file tools first before returning to this question.</p>
<p><br /></p>
<h3 id="the-how-great-tools-for-editing-and-managing-plain-text-files">The How: Great Tools for Editing and Managing Plain Text Files</h3>
<p>The great thing about working with plain text files is you can open them up on almost any tool. This makes it quick and easy to edit a document in the command line, on your phone or on any number of text or code editors.</p>
<p>While I listed “poorly designed” and “weak user experience” as one of the disadvantages to using plain text files and existing editors, this issue has started to change. In fact, due to the increased popularity of simple writing tool, there are a number of great plain text editors out there.</p>
<p>Here several I like:</p>
<h5 id="typora-one-of-the-best-looking-and-best-user-experiences-for-markdown-writing">Typora: one of the best looking and best user experiences for markdown writing</h5>
<p>Along with The Archive, Typora is my primary plain text editor.</p>
<p><img src="http://www.markwk.com/images/2018-resources/evernote-to-plaintext-exported-files-example.jpg" alt="" /></p>
<p>Typora provides both a great looking interface and smooth user experience. The layout and typography makes for a pleasurable appearance. Unlike most other editors, Typora hides the actual markdown as you write and shows you the preview of what you’ll see in the output. It’s much like a WYSIWYG (“What You See Is What You Get”) editor. You’ll need to get used to a few shortcuts to make the experience smooth, but once you get the hang of it, it really is a great way to write and edit plain text documents.</p>
<p>One of my favorite features with Typora is its ability to embed images. This allows me to include local and external images that are displayed seamlessly, much like other full-featured note-taking tools.</p>
<p>Typora is available for Mac, Windows and Linux and is under active development.</p>
<h5 id="the-archive-connected-plain-text-files-organizer-and-management-tool">The Archive: Connected, Plain Text Files Organizer and Management Tool</h5>
<p>Keeping your notes organized and connected is one of the principal challenges to using plain text files. The Archive provides one of the best solutions so far.</p>
<p>Basically, after opening a directory of files, The Archive feels much like a normal note-taking app but uses any directory of files. You can search and open notes by file name or find related notes by a tag. You can add sidebar shortcuts to tags or specific searches, much like you would in a tool like Evernote. It also provides a simple and clean interface for editing plain text files or option to open an external editor.</p>
<p><img src="/generated/images/2019-resources/the-archive-writing-tool-800-b1c5d8303.jpg" srcset="/generated/images/2019-resources/the-archive-writing-tool-400-b1c5d8303.jpg 400w, /generated/images/2019-resources/the-archive-writing-tool-600-b1c5d8303.jpg 600w, /generated/images/2019-resources/the-archive-writing-tool-800-b1c5d8303.jpg 800w" /></p>
<p>The Archive is attempting to solve the problem of connecting together notes into a robust knowledge base using wiki-style links and tagging. So, if you’ve migrated your notes from a system with existing tags, then it should provide much of the same functionality you had before.</p>
<p>Embracing the philosophy of plain text files and tool independence, The Archive’s explicit focus is helping you build a <a href="http://www.markwk.com/smart-notes.html">Zettelkasten or Connected Knowledge System</a> on top of your existing plain text files. We will explore several of these organizational aspects around plain text files below.</p>
<p>The creators of The Archive also provide a good intro to <a href="https://zettelkasten.de/posts/overview/">what is a Zettelkasten</a> and a list of other <a href="https://zettelkasten.de/tools/">compatible tools for a connected notes system using plain text files</a>. Most notable are nvAlt and DevonThing.</p>
<p>The Archive is currently only available on Mac and is under active development.</p>
<h5 id="1writer-my-preferred-plain-text-editor-and-search-for-ios">1Writer: My Preferred Plain Text Editor and Search for iOS</h5>
<p>There are a lot of markdown editors for iOS. You can view <a href="http://brettterpstra.com/ios-text-editors/">feature-by-feature breakdown of markdown apps here</a>. I’ve tried several and found 1Writer to be the most complete solution. The editor itself is solid, and the preview option is great. But the real distinguishing feature is file search and tagging. 1Writer makes it easy to find files with a certain tag, view recently changed files or search. At present, I find the search function superior and faster on iOS to The Archive on Mac.</p>
<h5 id="monospace-solid-plain-text-editor-for-android">Monospace: Solid Plain Text Editor for Android</h5>
<p>I don’t do much writing or permanent note-taking on Android. I mostly just jot notes into a standard note-taking app that I sync and process on the computer. That said, my preferred editor for Android right now is Monospace. It looks great and performs well. I also find Writer Plus to be another solid option.</p>
<h5 id="plain-text-editors-abound">Plain Text Editors Abound</h5>
<p>Considering how active the space around plain text and markdown editors is, I highly recommend you do your own research and testing on the best markdown and plain text editor for you.</p>
<h3 id="a-few-key-practical-organizational-principles">A Few Key Practical Organizational Principles</h3>
<p>In a previous section, we left a question on the table, <em>How to stay organized with plain text files?</em> This question might be shortened to just: <em>How to stay organized?</em></p>
<p>Personally, I’ve invested a lot of thought, testing, and writings to personal organization, and it’s been an important aspect of a productive and creative professional life. For me, the core of my productivity and note-taking systems have come from <a href="http://www.markwk.com/category/getting-things-done">Getting Things Done (GTD)</a>, The Organized Mind, Wikipedia and, most recently, the Zettelkasten Method, which I covered in <a href="http://www.markwk.com/smart-notes.html">A Book Review of How To Take Smart Notes by Sönke Ahrens</a>.</p>
<p><img src="/generated/images/2019-resources/organizational-principles-infographic-450-d439e00a1.png" srcset="/generated/images/2019-resources/organizational-principles-infographic-400-d439e00a1.png 400w, /generated/images/2019-resources/organizational-principles-infographic-450-d439e00a1.png 450w" /></p>
<p>A few practical principles have emerged:</p>
<ul>
<li><strong>Outcome Thinking</strong>: Think about what you want to accomplish. Every project, task or goal should clarify what you intend to accomplish.</li>
<li><strong>Establish Next Actions</strong>: Figure out what is the next best action is on something and then do it.</li>
<li><strong>Leverage External Systems</strong>: Our memories should not be the primary place where we store information or organize the things we need to do. Use a list, notebook, calendar or to-do list application to get stuff out of your head and into a system. In short, embrace tools that help you work best.</li>
<li><strong>Develop Processes</strong>: Processes allow you to automate how you do something and remove the need to think too much. Much of what stresses us out about work are open threads and unprocessed items. We similarly struggle early on in a new job or facing a new type of task. In these situations, it’s best to spend some time learning and experimenting with processes. Once you figure out processes and flows to deal with any challenge, then you start to remove the start and automate a best way of doing something. This apply to nearly everything, like inbox emails, requests, information and other stuff. Research how others are doing but be willing to use your processes too.</li>
<li><strong>Review and Reflect</strong>: Taking a step back, cleaning up and reviewing are critical to a robust organizational system. For me the best single thing I do is a <a href="http://www.markwk.com/data-driven-weekly-review.html">Weekly Review</a>.</li>
<li><strong>Do the Work</strong>: Assuming you have a system, tools and processes, then it should empower you to not be stressed out, find a zen-like state and focus on the work.</li>
</ul>
<p>These principles apply most directly to project and task management. The main tools I’ve used historically are <a href="http://www.markwk.com/gtd-with-evernote.html">Evernote</a>, Todoist, and a Calendar. I still use <a href="http://www.markwk.com/gtd-with-todoist.html">Todoist as my GTD-inspired task manager</a> as well as a way to <a href="http://www.markwk.com/task-tracking-with-todoist.html">track what I get done</a>. In place of Evernote, I now use plain text files to accomplish many of the same roles, like outcome thinking, goal planning, clarifying next steps and periodic reviews.</p>
<h3 id="the-how-best-practices-for-naming-tagging-and-linking-files">The How: Best Practices for Naming, Tagging and Linking Files</h3>
<p><strong>The core organizational idea behind my plain text files-based notes, organization or writing system is this: uniquely named plain text files; files connected together using manual tagging and links between files; and files grouped into a few targeted directories by either project or purpose.</strong></p>
<p>The only requirements of my notes and writing system are: 1. notes must have a permanent and unique identifier, and 2. they must be capable of being connected to other notes. The unique id is generated by a date timestamp, which by the passage of time means it won’t be repeated. Connections are provided through manual links between notes and by adding tags.</p>
<p><br /></p>
<h5 id="file-naming-convention-unique-creation-datetime-and-an-optional-keyword-or-short-title">File Naming Convention: Unique Creation Datetime and an optional keyword or short title</h5>
<p>All files should have a unique file name.</p>
<p>There are various opinions on how best to name your files. For our needs, setting your own title won’t work, because you might change it one day. It might not be unique, especially if you occasional create similarly titled notes.</p>
<p>It might not seem obvious why you want your file names to be unique. A unique identifier is responsible for keeping that note separate from other notes and allows for connections between files. With a unique and immutable identifier you can separate that piece of content from everything else and you can be sure links to it will go to the right place.</p>
<p>So, the current best practice to use a unique name based on the datetime when you create that file. This results in files like 201902111136.md or 201603302120.txt.</p>
<p>A program like The Archive or 1Writer can be configured such that new files default to this naming convention. Additionally, you can use a <a href="https://gist.github.com/markwk/86ad08822d7cfe47ed8533eb812a233d">bash script to automate creating unique plain text files</a>.</p>
<p>While I recognize the need for unique filenames and believe that the file’s creation datetime is the best starting point, I still find it useful to have an indicative name or keyword as well. Without a keyword in the filename, search because a challenge.</p>
<p>Personally, the best file naming convention I’ve found includes a unique name based on the creation data with an optional keyword or short title after. This results in files that look like this: 201812182000_neurofeedback.md, 201602261908_My_Studies_Meta-Note.md, or 201901101453_model_of_habits.md.</p>
<p>This ensures a unique identifier with the datetime so you can reference and link to the file elsewhere as well as making file lookups faster from memory.</p>
<h5 id="beyond-just-search-creating-direct-links-between-files">Beyond Just Search: Creating Direct Links between Files</h5>
<p>All notes should be linked to other pieces of information in your system.</p>
<p>Search and discoverability have always seemed key features for me when looking for a note taking and knowledge setup. But I’ve come to realize the the ease of search has resulted in significant cognitive overload later and a somewhat muddled way of organizing things overall.</p>
<p>With search we can often times can get to what we wanted, eventually. As <a href="https://zettelkasten.de/posts/search-alone-is-not-enough/">one of the writers on Zettelkasten notes</a>, search is like a shotgun. Sometimes it hits but it’s method is a wide blast and often times it’s just getting you a broad selection you still need to process through to find something. What we want is a sniper’s precision rather than a shotgun approach. Manual connections and proper tagging are the sniper and curator at its best.</p>
<p>My objective is to create a latticework of notes, much like how our brains work. What I’m building is a curated network of linked information. As such, we need consciously built connections for this to work.</p>
<p>There are a couple of practical ways to create connections between notes. If you are using a Zettelkasten tool like The Archive, you can use wiki-style links with double brackets. You can also use a full-qualified link to the file on your file system. Personally, I use both methods. In both cases I’ve literally connected pieces of information into a network that makes it easy to “dance” through referenced notes and related topics smoothly.</p>
<h5 id="tagging-creating-shared-flexible-groupings">Tagging: Creating Shared, Flexible Groupings</h5>
<p>I believe all tags must also include at least one tag.</p>
<p>Tags are another form of organization. Unlike a folder or directory, tags are flat and non-hierarchical. Multiple notes will all reference a single tag. You can then do a search on a single or multiple tags to find all tagged content. Like a link, tags are a manual, curated process where you build out key connections in your network of notes.</p>
<p>Tags can be quite flexible, allowing you to build flexible groupings. Applying tags to certain notes allows you to reference a topic that may not even be mentioned in the note’s text.</p>
<p>While tagging can seem easy initially, as your system grows tagging can admittedly become rather huge and overwhelming, especially if you have dozens or hundreds of notes sharing the same tag. Tags remains one area where you likely should consider specialized tags based on the type of note. For example, a comprehensive summary note might be tagged #SummaryNote or a blog draft as #DraftWorking.</p>
<p><br /></p>
<h3 id="notes-for-what-how-to-stay-organized-with-plain-text-files">Notes for What?: How to Stay Organized with Plain Text Files</h3>
<p>If you are a regular writer or note-taker, then it’s likely you have lots and lots of files and notes. In my own case, <a href="http://www.markwk.com/migrate-evernote-plaintext.html">when I migrated off of Evernote</a>, I ended up with 11,278 note files. That’s so many files that I couldn’t even use certain basic search functions on the command line. And since I had many different types of notes from personal writings and project notes to collected articles, journal entries and random thoughts, it was not immediately obvious how to organize all of these files.</p>
<p>Sure, everything was tagged, but where should everything go?</p>
<p>We often conflate working on our productivity system with actually being productive and working. They are not the same thing. In fact, how you stay organized should serve the outcomes you want to achieve (and not the other way around). As such, we need to remember that <strong>our file and notes system are for two purposes: organization and creation/learning</strong>.</p>
<p>This means we should develop and maintain a system that help us towards those two roles of planning and staying organizing and creating and learning stuff.</p>
<h3 id="fallacy-of-the-collector">Fallacy of the Collector</h3>
<p>One of the key things I realized early on when I switched to plain text was that a lot of my notes didn’t serve either of these roles. Instead they just helped me collect stuff.</p>
<p>As a self-tracker and documenting guy, I love collecting stuff and tracking different aspects of my life. For example, automation tools like IFTTT and Zapier can make it seamless for me to pull in links, articles and clippings from tools like Todoist or Instapaper. Evernote and most note-taking tools also make it a tad too easily to be used for miscellaneous collecting. Once all of my notes were into plain text, I discovered how much of it was just collected stuff.</p>
<p>The key realization here is that your plain text files system should not just be another collection system. In fact, collecting and aggregating should be a minor aspect of what these systems should do. What our system should help us do is to learn, connect and create and to stay organized.</p>
<h3 id="note-types-different-notes-for-different-needs">Note Types: Different Notes for Different Needs</h3>
<p>One helpful step in the direction of better organization was clarifying the type of notes I have and generate. At present I have six primary types of notes:</p>
<ol>
<li><strong>Impermanent, Fleeting Note</strong>: These are the quick notes notes you capture while reading or learning. You jot-down them in a physical notebook while reading a book or article or watching a video online. A thought may come on a walk or on a commute that I record in an audio file or my phone’s notepad app. They are often just a rough concept you want to capture immediately. The intention isn’t permanence since eventually you process and transfer these fleeting notes into something more permanent, like an insight note.</li>
<li><strong>Permanent, Insight Note or Smart Note</strong>: A smart note is a distilled, atomic idea you create in writing. They explain something in a concise and precise expression. You can’t just copy someone else’s way of putting it. You have to express in your own words and thoughts. The notes should be connected to other notes, tagged with key concepts, and include references to the related source material if possible. As I put it in <a href="http://www.markwk.com/smart-notes.html">A Book Review of How To Take Smart Notes by Sönke Ahrens</a>, “these permanent notes strive to be the stepping stone between our fleeting notes from what we read and our eventual creative production.” These notes can be used as both a form of elaborative learning and to provide processed knowledge notes for on-going use in your thinking and creating.</li>
<li><strong>Project Notes</strong>: These are meta-level, organizational notes. They can used for both professional and personal projects, initiatives, goals, studies, etc. These notes often include lists, plans, to-do’s, personal reflections, and references to follow up on. It might be a process journal for initial drafting, planning, and thinking through. They often serve as a way to reflect <em>about</em> the work. The work itself is covered in actual writings, insight notes or elsewhere. For me project notes have a list of next actions and links at the top and a journal of progress below.</li>
<li><strong>Actual Writings (or Creation Notes)</strong>: These are the actual things you create. This includes drafts, manuscripts, scripts, dialogues, published writings, or whatever you target output is. This is where you do the work. For me, these are blog drafts, chapters, and even markdown slides.</li>
<li><strong>Summary Notes (aka Structure Notes)</strong>: Sometimes called Outlines, Index Files and Theme notes, these kinds of notes serve as an entry point into a specific topic or area. It might be an early draft or a way to conceptualize how different pieces of knowledge fit together. It could include a raw unsorted list of links to the various notes related to that topic or, as I prefer, a more sorted, organized format. In general, these structure notes service as a high-level organizational tool making it easy to build up knowledge over time and an intermediary step between insight notes and a manuscript or other creation. Depending on the project or topic, a summary note might also be be a nice complement a project note showing you what you have and gaps that you may want to work on.</li>
<li><strong>Collection (or “Collector”) Notes</strong>: These are random assortments of links, highlights, clippings, and quotes from books, articles and online material. I find that they are best kept separate to avoid polluting these other notes with the unprocessed stuff. It is nice to have them available when you want to check a reference or find a specific quote from a book or article. Frankly, as a long-time collector of notes rather than a taker of a notes, these collected bits should not be the goal of your system.</li>
</ol>
<p>With these note types, I’m now better at staying organized and at using certain notes towards an intended purpose. This is better than what I previously had which was one mega note that mixed quotes, insights, reflections and collected study materials.</p>
<p>For me the core distinction worth reiterating is between project notes and insight notes. Insight notes are atomic ideas distilled largely in your own words on a single topic or idea. Project notes are the meta-note about an area I’m working on. These could be goal, a field of study or some other largely unified area.</p>
<p>For example, my reflection on the process for a goal or project is best separated from the work itself, like learning notes or writing on that project. This is actually how writers like John Steinbeck worked. They had the book they were writing and a separate meta or process journal as well as letters to friends describing the effort and challenges of that project.</p>
<p>Similarly, it doesn’t make sense to mix my actual study notes with the meta-level note on the goal-striving related to those studies. Their roles are separate so it makes sense to split up the notes.</p>
<p>This realization has lead to the primary way of separating notes into directories.</p>
<h3 id="the-where-organizing-separate-directories-and-grouping-your-files-and-notes">The Where: Organizing Separate Directories and Grouping Your Files and Notes</h3>
<p>Now that we’ve defined our core note types, we can then think about where those files should go and how to organize them.</p>
<p>While we’ve set a few key requirements around filenames and organizational aspects and with tags and direct links, we haven’t talked about how best to group and organize all of your notes.</p>
<p>Some argue in favor of splitting notes into different projects and areas, while others believe it is best to keep everything in a single system, like a Luhmnan’s Zettelkasten. In general it is best to not split your notes into too many subdirectories or sub-projects. this leads to notes getting buried and lost.</p>
<p>In view of my goal of a networked system of notes, my notes work best all together, linked through flexible structures like tags and direct links.</p>
<p>The one exception is your “collector notes,” which I’d say are best kept entirely separate. These notes are other people’s words, pollute search and distract from the importance of putting things in your own words.</p>
<p>For the other notes, keep as many of the notes as you can together and use tagging as main way of staying organized.</p>
<p>In terms of how many folders and directories to use, it depends on you, but I find that a balanced approach works best for me. At present, I have four folders or file directories:</p>
<ul>
<li><strong>My Archive</strong>: This is where my writings, insight or smart notes, and summary notes go. This is where I do my work. Since writing involves using my insight notes, it makes sense to have them in a unified setup.</li>
<li><strong>My Project Notes</strong>: This is where all of my notes on projects, initiatives, goals and progress logs are kept. I typically have a separate notes for each project, learning area, or goal. Inside I have a list of next actions and important links at the top, and at the bottom, I have a process journal recording my progress and reflections.</li>
<li><strong>The Collected Stuff Directory:</strong> Beyond these core three, I also have a directory entitled <strong>z_collected</strong>. This is where clippings from articles and kindle readings go. It’s also where I have stored my legacy notes, unsorted stuff, and past articles. For example, I use some python code to take Kindle clippings from each book I read and convert them into a separate note. I have another script that takes highlights from Instapaper articles I read too. These are nice things to have but best kept separate.</li>
</ul>
<p>Arguably I could go and either combine all of my notes into a single directory or further refine my notes into additional subdirectories. I find that the current separation works well for me.</p>
<p>It separates out three key areas of my life and work. I have notes about my projects and goals, which serve meta-level organizational objectives; I have my writings and learning notes in a single knowledge and creation base, which allows for an organic and connective approach to learning and writing; and I have my coder notes to follow my technology work. Overall, this division is a good conceptual division too. It also makes it possible to track the evolution of each of these areas separately.</p>
<p>There is a lot more that I could share related how I take and manage specific types of notes. For now, I’ll have to leave these topics for future posts.</p>
<p>My main advice here is to realize that notes and writing serve different purposes. You generally want all of your notes together and to use manual connections and flexible tagging to serve as your primarily organizational methods. When developing your own setup, define specific note types towards your creative and your organizational goals.</p>
<h3 id="conclusion-building-a-latticework-of-notes">Conclusion: Building a Latticework of Notes</h3>
<blockquote>
<p>My goal was not to replace Evernote but to evolve my system. My objective was to reconsider my organization, writing, and how I capture, process and use bits of information productively, creatively and meaningfully.</p>
<p><em><a href="http://www.markwk.com/migrate-evernote-plaintext.html">Post-Evernote: How to Migrate Your Notes, Images and Tag into Plain Text Markdown</a></em></p>
</blockquote>
<p>When I migrated all of my notes off of Evernote and into plain text, I wasn’t looking to repeat what I had done before: Create a convenient, grab-bag collection of notes, articles, writings and project ideas. Instead, I wanted to “evolve my system.” The current working result is what I call <em>a plain text life</em>.</p>
<p>At its core are simple plain text files that are future-proof and flexible. I write my files in markdown. These files can be opened up on any device and edited on any number of text editors. Currently my two favorites are Typora and The Archive.</p>
<p>The real challenge has been and remains: How to stay organized? This isn’t merely a question of information management and note-taking but of learning, memory and human creativity.</p>
<p>So far, a few practical, organizational principles have emerged.</p>
<p>First, all files need a unique name. My current preference is a combination of a unique datetime and an indicative keyword or title. This second feature enables easily file look-ups.</p>
<p>Second, all files should be connected to other files through direct links as well as an evolving system of tags. While it’s possible to have notes that aren’t linked to others, what we are building should hint at the flaw in this. It would be like having a single neuron storing a single memory but not connected to anything else. The memory neuron would be useless since it wouldn’t be connected or retrievable. It’d be a memory you could never get to. Notes should aim to be interconnected.</p>
<p>Third, strive to keep all files together in as few directories as possible. This is both practical and conceptually. Practically, fewer directories forces you to develop an interconnected network, rather than sub-divided directories. Files come into contact and become more meaningful when interlinked. Conceptually, I define and divide things using a different note types and a few directories according to my primary use cases: learning/writing, organization/projects/goals and coder notes. This corresponds with how the plain text life applies to my work as such as a writer, engineer, and life-long learner. There are a lot of ways to organize information and knowledge and I encourage you to explore your own way.</p>
<p>This final point leads me again to ponder the question of organization. It’s a question that I suspect will continue to haunt me going forward too. To repose the question: <strong>What is the best way to organize information, knowledge or our creative work, especially in view of our goal of creative output?</strong></p>
<p>In writing this post, I realize that at times I may have come off as a “comprehensive,” “systematic” or all knowing. This was more about my trying to encompass and write about a range of topics, rather than some claim at the definitive answer. Learning and creativity are complex topics. Both are still being actively studied by researchers today and much still needs to be explored before we can claim a definitive understanding of how thinking works.</p>
<p>That said, at least two key aspects from the research stand out to me when it comes to learning and creativity. Those are network and complexity. Basically our brains work through a network of connections. <a href="http://www.markwk.com/science-of-learning.html">As we learn</a>, we build connections between chunks of information. Complexity defines that fact that we can never reduce it to its individual aspects and instead properties and new aspects emerge from that complexity and network. In our brains, as a complex network of ideas emerge, these existing and emergent connections enable us to learn, think and create. We are the product of complex, internal neurological networks.</p>
<p>In this same spirit, my notes system strives to emulate and evolve as a complex network of notes too. The organizational structure holds similar assumptions such that notes should be linked together and exist as part of a complex network of interconnections.</p>
<p>This complex, connective approach to notes means we must largely move beyond any attempt at hierarchy. Notes will never fully or completely be ordered and organized. This includes knowledge and smart notes, written drafts and organizational project notes. We must accept them as a network since it’s from this network where emergent, magical things happens, like thinking, learning and creativity. In our notes and writing systems, we must embrace the network and endeavor to write and create notes much like how our brains works: as a complex <strong>latticework of connections</strong>.</p>{"image"=>"mark_koester_headshot_square.jpg", "greetings"=>"Hi! My name is Mark Koester. I'm a data-centric technologist based in Venice, California."}“The power of the unaided mind is highly overrated. Without external aids, deep, sustained reasoning is difficult.” Don Norman, professor and author of The Design of Everyday Things