Mark Koester

Personal Blog. A Data-Driven Life. Minding the Interstitial Spaces since 2007.

Tracking Your Writings and Note-Taking

How to Track, Analyze and Understand A Life in Writing

“For years I have looked for the perfect pencil. I have found very good ones but never the perfect one. And all the time it was not the pencils but me. A pencil that is alright some days is no good another day.”

John Steinbeck, “Journal of a Novel: The East of Eden Letters”

Can we track what we write? If so, how? And what can we use our writing tracking for?

As a self-tracker and an enthusiast of the data-driven life, I track a lot of my life, but as a writer, I find the tracking options rather limited. There are tools to log my writing time and how words I typed, but there is nothing that allows me to keep a complete history and data on all of my notes, drafts and final creations. I wanted a way to track what I write, not just my time or word count.

Fortunately, since I migrated off Evernote, I now write in plain text files. Plaintext files are a file format that is future-proof, flexible and portable. They are also trackable, and I am able to track my life in writing.

Using git, a popular way to manage software development, I have assembled a self-tracking method for keeping a complete history of my files, including each and every change I make, and for logging daily statistics on words added, files changed, and more. In short, with plain text files, git and a few automation scripts, we have a comprehensive and robust method of tracking our writings and notes.

In this post, I want to share how to track your writings. By using plaintext files, git and a few scripts, I’ll show what it takes to record a complete history of your notes into git and also collect some high-level statistics of daily changes. Since it is important to know what data we are getting and the potential insights we can get from the data, I’ve also provided a starting point for some data analysis on that tracking data.

Hopefully, by the end of this post, you’ll have mastered the basics of managing and tracking your writings with plaintext files and git and equiped yourself with a way to comprehensive way to track your writings and notes in the future!

NOTE 1: You can find the code for this post at Writing Tracker.

NOTE 2: For a more in-depth dive into writing and note-taking check out The Plaintext Life: Note Taking, Writing and Life Organization Using Plain Text Files, and for a step-by-step tutorial on migrating off of Evernote to plaintext files, see Post-Evernote.

The Plain Text Life: Note Taking, Writing and Life Organization Using Plain Text Files

“The power of the unaided mind is highly overrated. Without external aids, deep, sustained reasoning is difficult.”

Don Norman, professor and author of The Design of Everyday Things

External aids, especially writing, are the key to sustained learning and a creative life. The goal of your productivity, writing or note-taking systems should be to enable you to think clearly, stay organized, learn, and create. They should augment your ability to reason, to develop connections across knowledge, and produce a targeted output.

There are a lot of tools that can help you in this pursuit. We live in a world of nearly endless options for productivity and writing software. Personally, I’ve tried many. But sometimes the best solution is one of the simplest and oldest. For me, that solution was “downgrading” to plain text files as my primary means for note-taking, writing, knowledge management and life organization.

Rather than a fully featured notes or writing tool, I now have a bunch of plain text files and a lot of them. The files themselves are simple, can be edited on any system, and are future-proof. I write in markdown. I use plain text files not only for my writings, study notes and note-taking but also for my goals, organizational, project notes.

I call this method and approach a plain text life, by which I mean a note-taking, organizational, and writing system based on plain text files.

The files and notes themselves have been intentionally designed to be “organized” as network of information. Practically-speaking this translates into notes stored in a few directories, tagged, and connected together using a links. The end result is a loosely-coupled web of notes. It’s evolving, has emergent properties and is trackable too.

This setup helps me focus on what matters: writing and keeping my projects, ideas and thinking organized and interconnected.

In this post, I want to share my take on the “plain text life” and how using plain text files and combination of tools, best practices, and organizational principles can unlock a powerful and efficient framework for writing, thinking, note-taking, project management, goal tracking, or whatever you are working on.

This post is divided into four sections with four questions:

  • Why plain text files and what are the current limitations?
  • What tools can I use for writing and managing plain text files?
  • How to stay organized?
  • What are my notes for and how to organize towards my creative, learning or organizational goals?

At its core the plain text life is just a just a bunch of text files, but hopefully you too can assemble a powerful framework for staying organized, writing, learning and creating.

Let’s get started exploring a plain text life!

NOTE: For a how-to post on migrating off of Evernote, check out Post-Evernote: How to Migrate Your Notes, Images and Tag into Plain Text Markdown.

A Year in Data: 2018

What can one learn from a year in data? What lessons and observations can be drawn from tracking a life?

Before jumping into the analysis, here are three “teaser” data visualizations and questions that data helped me notice and answer from the last year:

The majority of my manually logged time went to personal projects in 2018, which is great. I still maintained decent client, freelance income, but my focus remains on own stuff and businesses. . If these are your goals, this is where your time should go. But happened in June and August? Both months saw a decent time log drop. Both months had me speaking and attending more conferences as well as more travel, including a week long trip with friends to Tibet and aroudn Sichuan. My sleep tells another story though too

I slept on average 7.23 hr per night in 2018, but what happened starting in mid-June? Why did my sleep average drop? World Cup! Like many I stayed up later and drank a bit beer more as I enjoyed an epic month of football action. Ironically other areas also dropped too. I wrote less and had less workouts. See below for the specific charts.

I had a relatively consistent year of running in 2018, until the last quarter of the year? What happened? Suddenly my run training load dipped signficantly. This was due to a series of minor to a bit more major injuries, and I couldn’t train. What was the cause? Below I examine a series of other tracked data points and currently my working hypothesis is one contributing factor was a decrease in both mobility/stretching and strength training.

These are three data visualizations I created using simple but regular self-tracking and data collection data visualization using QS Ledger. If you are interested in creating similar charts or becoming more data-driven in the year ahead, signup for my newsletter to receive early access to my course, Google Data Studio for Quantified Self and the Data-Driven Life in early 2019.

Now on with the show!

My Data-Driven Annual Review

Annual reviews are a staple of goal-directed people, and it’s something I like to do myself. But in view of my central thesis that data can be used to improve human lives, I like to go a bit beyond just nostagic reflection or hopeful and strategic aspiration. I like to fuel my year in review with data.

Let me tease out my core belief briefl here, We have now enough cheap technology, sensors, small computers (i.e. phones and wearables) and cloud storage to enable both tracking a lot of areas and storing a huge amount of data on a life. This activity is often grouped under monikers like quantified self, self-tracking, personal infomatics, or the data-driven life.

Basically, it’s people collecting data on a life and trying to use it. Quite simply, I can and should use data and self-tracking so our lives can be better understood and improved. This is data-driven personal development 101.

So, it’s with those goals and hopes in mind, I’m happy to share the latest edition of my year in data!

This year’s report includes a look at how I wrote, my computer usage time and project time, my trends in tasks completed, books and articles read, my fitness efforts, and a bit of my photos taken on my phone. For health, I place an emphasis on running but also have some general health and wellness data on sleep and HRV too. Obviously there is a ton of data one might use to slice and dice and tell your year in review story. I choose these since they matter to me and are relatively accessilble to anyone to track.

One bonus this year is that, unlike last year’s year in data, I’m also open sourcing and sharing all of the code I used to create these graphics. So for anyone with a few technical skills and some time, you too can build your own data-driven year in review. Morever with a few tweaks you can change the look and feel or make additional observations and analysis too. You can find the code at QS Ledger, and I’ve shared a few of the specifics on how to do this at the end of the post.

Hopefully by the end of the next year, I’ll have a web or mobile app to make it even easier for anyone to do and even create a full-on book about your year… using data.

Let’s take a look at the story of my year in data!

(NOTE: While outside of the scope here, I have also done some work on a deeper statistical analysis and machine learning approach to using this data. For now I’ll have to leave that topic for a future write-up. )

What Should You Be Tracking in 2019?

Measuring a Life, Understanding Progress, and Checking Your Status Towards a Goal

Tracking and personal data can and should be part of how you pursue goals, develop better self-understanding and optimize self-improvement.

As a new year arrives, many of us often set new goals and resolutions. During time-triggering events like a birthday or a new month or year, we declare what we want to change and attempt to build a new habit or reach a long-desired goal. A lot of those will fail. By some estimates over 90% of new years resolutions fail.

We think a lot about what we want to achieve. But how often do we think about the process underlying how we achieve or even how to measure our progress towards those goals?

While a lot of jargon terms get thrown around, at its essence self-tracking, quantified self, personal informatics, or whatever you call it can be defined as the activity of measuring or documenting something about your self. In turn, I find it’s best to frame this tracking towards either better self-understanding or optimzed self-improvement. So, when it comes to goals, tracking data can serve as a feedback mechanism for understanding a specific area like health or productivity or as a gauge to measure your progress towards an objective goal.

In this post, I want to share what I’ll be tracking in the year ahead, but I also want to argue why I think tracking is a useful and meaningful activity today.

In the first part, I’ll share a few reasons why many people track and why personal data collection is such a valuable pursuit today. To cut to the chase, the main reason I find tracking beneficial is that it is an enabler for better self-understanding and empowered self-improvement. But the only way tracking can be an enabler is if we go beyond just tracking and data collection and start to engage with our data. That’s why I believe data engagement is so important. You don’t need to be data scientist to put your data to use.

In the second and longest part, I’ll layout what I’ll be tracking in 2019, including the specific area and technologies I use. I’ll also share three ways I engage with my trackind data through a weekly review, personal data dashboard and goal check-in’s.

In conclusion, I’ll briefly share four areas I think everyone should track and how tracking and personal data can align with your goals.

Let’s get started looking at what you could be tracking in the year ahead!

My Year in Book Reading: 2018

I’m striving to be more conscious about what I read, how I learn from books and articles, and how I apply those lessons to my life, my work and my projects. So, in the spirit of that, I’m continuing a tradition I started in 2017 by offering up my year in reading for 2018.

Unlike my 2017 edition, I’m also including my article reading soo, which I track using Pocket at the start of the year and Instapaper since August. As an added bonus, I’m also sharing a few ways to do your own data-driven year in reading using some open source code I’ve created or you can sign up at XXXXXX to use the online tool once it is available.

Topline Numbers

2018 was good year for me and reading.

Book Reading Numbers: I read 16,749 pages across 60 books. My average rating on books I rated was 3.86. Compared to 2017, I read fewer novels and more non-fiction. My non-fiction reading tended a bit more towards science and academic research than previous too. My biggest month of book reading was May in which I finished 8 books and over 2,000 pages.

As a Kindle reader, I’m fortunate to have a whole log of my highlights too. In 2018, I collected XX highlights from books I read.

Article Reading Numbers: By my account, I read 1785 articles. While I tend to bias my reading towards long-form books, I also did my fair share of article reading. Since I track my article readings with Pocket and Instapaper, I more or less know how many articles I read (or at least the articles I read and tracked via one of these apps).

Below is my data visualization of my year in reading book, the top books I read and my key takeways, my year in article reading and a few favorites. Finally, I conclude with a couple of thoughts on my reading habits and goals for the year to come.

Learning to Learn: On the Science of Memory and Effective Learning

One of the most useful and interesting things I learned in the last year was not so much a specific subject or skill but a meta-skill. I learned how to learn.

Learning is a critical skill for everyone. Whether you are a full time student, working full-time or retired, learning is an important aspect for people living in a constantly changing and complex world today. If your full-time job is studying or just spend a few hours a week learning new things like I do, certain study skills and learning techques can help you to learn more effectively. Additionally possessing an understanding of what is learning and how our memory works will help you understand why certain learning techniques work and how to adapt your lifestyle and health towards improved learning, thinking and creativity.

As a life-long and constant learner, I’m always learning new things. I have a love for foreign languages, technology and the social sciences, like philosophy, psychology and sociology. As my blog writings document, I’m often writing about the things I’m learning too. Ironically I’ve come to realize that this act of blogging what I learn (or giving or teaching, for that matter) is one of the better ways to learn. Elucidation forces you to engage with a topic, figure out gaps in your knowledge, connect it to what you know, and put it into a meaningful ordr or story. Unfortunately some of the other ways I like to use when learning like highlights and summarizing are not great learning methods and even promote something called the illusion of competence.

To the cut to chase, the best course I took recently was Learning How to Learn, a free course available on Coursera. I highly recommend it to everyone. The course is taught by Professors Barb Oakley and Terry Sejnowski. Oakley focuses more on the practical aspects of memory and thinking, while Sejnowski digs more into the neurology and brain science behind what happens in memory and learning.

Along with a lot of great material, links and resources, the course offers an example of “practice what you preach,” meaning the course provides great analogies, metaphors and visualizations of the more complex concepts. These funny examples and metaphors provide a way to “hook” new concepts into your memory and make remembering key concepts easier. Additionally, in view of how research reveals the importance of testing, intermingling, and elaboration to learning, it’s no surprise you get several ways to engage with the learning in the course by taking quizzes, doing homework, and connecting concepts together. My big takeway is I know that I can get better at learning and now know many ways how to do it.

In this post, I want to share some of the lessons I picked up from learning how to learn, both from this course and from additional research. In the first section, I’ll argue for why everyone should learn how to learn and some of the benefits to understanding your learning apparatus. After that, I’ll share a few resources I use to learn more about learning. As a technologist, I’ll also share some of the tools I use to learn languages, new theories and coding. Finally, in the longest section, I’ll offer a summary account of what is learning and how memory works as well as a few things we can do in our lives to optimize both.

Hopefully by the end I’ll have convinced you to improve your learning and your understanding of learning as such.

Let’s get started looking at one of the most important meta-skills we all can improve: our ability to LEARN!

Post-Evernote: How to Migrate Your Evernote Notes, Images and Tags Into Plain Text Markdown

14,147. That’s the number of notes I had in Evernote.

A few weeks later, only a few thousands notes remained in Evernote. In their place, I now have 11,278 plaintext files and a completely new way to write, learn and organize my work.

Over the years, my personal usage of Evernote had grown to cover more than just note-taking and journaling. I had come to depend on Evernote as the “Swiss Army knife” of my productivity tool kit. For example, I had used Evernote as my task manager, Evernote as a read-it later app like Pocket or Instapaper, and even Evernote as a sales and networking CRM. Evernote’s mission to “capture everything” had largely became how I used the tool.

Unfortunately, a few cracks started to appear with Evernote and my usage. First, my Evernote notes had become a bit of a monster, both conceptually and organizationally and in terms of the total number of notes. I felt a desire to to refine my note taking process and to slim down the number of notes I had. Second, Evernote as a product and company had seen better days.

The problems with Evernote as a company and as a product are not really the point of this post. But a quick summary of Evernote problems will often include: pricing changes, feature bloat, privacy around your notes, significant corporate changes, lack of product additions, and poor product performance (at least for me on Desktop).

Personally I rarely had much of an issue with the product or paying for a great product, like Evernote. But these concerns had built up over time and formed into on-going questions like: What’s going on with Evernote? Is it time to leave? How can I migrate? What should I migrate to?

A couple of months ago I finally decided to explore some Evernote alternatives and how I might migrate my notes. There are some solid Evernote replacements but I elected to switch to my notes to plain text files. Though Evernote’s corporate and product issues played a part in my decision too, my shift to plaintext files was less a rejection of Evernote, and more of a push to change up my way of organizing and working. To be clear: My goal was not to replace Evernote but to evolve my systems.

Migration is not an insignificant undertaking. Evernote makes your life easy for collecting, jotting ideas and then finding your old notes and documents later. If you have been a heavy user of Evernote, you likely have hundreds, if not thousands, of notes. Migrating to a new system is a time-consuming effort, and you still need to consider and adjust to your new way of working too.

There are several ways to migrate off of Evernote and onto another tool. One of the easiest note-taking tools to import into is Bear, a Mac/iOS markdown notes app. Lifehacker has a decent, though somewhat dated, post sharing several approaches for migrating to Microsoft’s OneNote, Apple Notes, or Simple Notes. Unfotunately none of these approaches work for migrating off of Evernote and onto plain text files. Even the best script, Ever2Simple, won’t keep your images, tags and meta-data when migrating to txt files. Losing so much information from my notes was a non-starter for me and forced me to find a new approach.

Fortunately, as I’ll show in this write-up, with a couple of steps and a combination of tools and scripts, you can effectively export your entire collection of notes out of Evernote and into markdown plaintext files. Most importantly, you can also still preserve the essentials of your old notes like images, tags, and even metadata like date created. Yoou can also maintain your legacy Evernote links between notes.

In this post, I’m going to show you how to migrate your notes out of Evernote and convert them into a collection of plaintext files in markdown. I’ll provide be providing a step-by-step guide to exporting out of Evernote and and processing into a format that you can open on any markdown editor. Additionally we will be sure to keep the images, links and meta for your original notes. Along the way, I’ll share some tips and my way of doing it too. At the end, I’ll conclude by briefly sharing a bit more about why I left Evernote and a few aspects of my new plain text life.

Let’s get started migrating our Evernote Notes!

The Power of Systematic Notes: A Book Review of How to Take Smart Notes by Sönke Ahrens

The first step in nearly “every intellectual endeavour” is to take a note. Writing notes is critical for how we learn, develop ideas and ultimately, create, and if you want to become a better writer or creative of any type, you need a better system and process for your notes.

Those ideas (take smart notes and build a connected, personal system of smart notes) are the central arguments of “How To Take Smart Notes” by Sönke Ahren, a book I recently read and have become somewhat obsessing over. Inspired by Niklas Luhmann (1927-1998), a well-known German social scientist and his method for managing his research and writing, Ahren explores how to be more productive, creative and organized using a system of deliberate note taking.

With over half of doctoral dissertations going unfinished (Lonka, 2003), Ahren’s main focus is on the organizational and creative problems of academic and nonfiction writers, and while the target audience is thesis writers, the lessons go well-beyond academia. I’d even argue that this provides one of the missing pieces to David Allen’s “Getting Things Done” method of productivity (Allen, 2001).

Throughout the text, Ahren argues for the importance of developing a special habit of note-taking and creating “smart notes.” Smart notes are a form of “learning through elaboration”, meaning we learn by putting complex ideas in our own words and by connecting them to other ideas. Smart notes are not just another way to collect stuff; their aim and goal is to foster and support creative and innovative output.

Based on these permanent, insight notes, we assemble a “knowledge management system” (my term) that he calls in German the Zettlekarten or in English the slip-box. It could also simply be called an archive. Ahren goes on to provide a tactical guide for developing and leveraging this interconnected knowledge system of smart notes throughout any creative project and ideally throughout life in general. Since smart notes form the nexus for what interests us, our organized thinking and on-going discussion questions, it’s both fodder for thought and where our writing should beginning.

Though technical and very specific at times, the book was a highly enjoyable read as Ahren journeys through processes underlying human learning, thinking, productivity and creativity. I highly recommend it for anyone interested who regularly writes (whether fiction or non-fiction) and for anyone who strives to better organize their knowledge and pursue innovation and creativity in any project.

Here are a few of the book’s key points that struck me in my reading and that I’m hoping to bring into my own learning and creative processes.

Towards a Science of Goals: Goal Setting as a Key Influence on Performance

“What you get by achieving your goals is not as important as what you become by achieving your goals.” – Michelangelo Buonarroti, Renaissance artist

In spite of our best intentions we fail at a lot of our goals. According to some estimates, a mere 8% of New Years’s Resolutions make it to the end of a year and nearly 80% have already failed or been abandoned by February. When it comes to academics and doctoral theses, over half will never be finished too (Lonka, 2003).

What causes such a high fail rate on our goals and can we do better?

While the idea of “s.m.a.r.t.” goals is often the first that comes to mind when you think about goals, there is an actual field of psychology dedicated to the science of goals. It’s well-researched and provides some very actionable approaches on how to better set and pursue our goals.

Started in the late 1960s, goal setting theory (GST) rests on its core claim that there is a relationship between goals and performance and that having a goal modifies how we behave. Though this idea might seem obvious now, this theory broke with the behaviorist tradition that interpreted much of how we behaved through either biological drives or rewards/punishments.

Goal Setting Theory attempt to explain how performance and motivation are affected by goals. One of the chief and earliest realizations of GST was the benefit to setting specific goals (over having no goals or vague “do your best” goals). It also found that there is a linear relationship between how difficult our goal is and how much better our performance is. To put it simply, the harder the goal, the higher the performance.

Subsequent research into goals has revealed many important aspects and key mechanisms on how to better plan and manage goals in companies, organizations and on a personal level. It has also crafted a powerful explanatory and actionable model called the High Performance Cycle (HPC).

In this post, we will be looking at the science of goals. While much of behavior is still viewed by many through the optics of biological drives or rewards/punishments, Goal Setting Theory (and other cognitive research on multiple goals like Goal Systems Theory) provide a much richer model for how goals function. This research indicates that two key things: 1. goals modify how we behave and 2. how we set goals affects aspects of how we perform and even how we feel.

As individuals and organizations, we can do better in pursuit of our objectives by learning the science of goals and applying its key lessons to how we set, track and manage our own goals!

NOTE 1: This is a fairly long and detailed post. If you want to skip the theory and just get to the applicable lessons, see the section below entitled, “How to Set Good Goals (according to science)”

NOTE 2: This post is part of a series on the science of goals. Also check out Goals as a Multi-Stage Pursuit.

How to Create a Time Tracking Dashboard Using RescueTime, IFTTT and Google Sheets

RescueTime is one of my favorite ways to track my life. It’s a great passive way to know where your time is going on your computer. But how to collect your data and what to do with all that data once you get it?

Increasingly I’ve been using various automation services as one of my data collection methods. While you can use manual exports or code like QS Ledger to collect data from different tracking services, an automation service like IFTTT can automate the data collection. That way all of your data is stored into Google Sheets for easy access and even simple data visualization and data analysis.

Once your data is in Google Sheets, you can leverage custom functions and App Script to process and prepare that data. In turn, the ubiquity of Google Sheets means it’s easy to then pull data from there into your favorite visualization tools like Google Data Studio, Tableau or

I see a lot of value in time tracking and time data. Personally, I started to track using time tracking tools when I become a freelance developer several years. Over time I discovered how time tracking made me conscious of my time usage, I learned to use time data in my weekly reviews and even explored a year of time tracking too.

In this post, I want to walkthrough setting up a simple time tracking dashboard with RescueTime, IFTTT and Google Sheets. First, we will use IFTTT for data collection from RescueTime into Google Sheets. We will then leverage some code in Google App Script to process and prepare our data. We will use some custom functions in Google Sheets to create some time dimensions from our date field. Finally, we will use some simple pivot tables and charts to do some personal data analysis. We are turning our tracking data into improved self-understanding.

The goal of this post show you some advanced functions for data processing inside Google Sheets. You’ll learn how to add and leverage custom code with Google App Scripts to extract information and do calculations. I’ll show you how to use array formulas to process columns of data in bulk. Finally, once we’ve done the hard technical work of data processing and data preparation, you’ll discover how easy it is to do some personal data analysis using pivot tables and charts and graphs.

Hold on to your spreadsheets. Let’s get started with some advanced data processing with Google Sheets!