Minding the Borderlands

Mark Koester (@markwkoester) on the art of travel and technology

Post-Evernote: How to Migrate Your Evernote Notes, Images and Tags Into Plain Text Markdown

14,147. That’s the number of notes I had in Evernote.

A few weeks later, only a few thousands notes remained in Evernote. In their place, I now have 11,278 plaintext files and a completely new way to write, learn and organize my work.

Over the years, my personal usage of Evernote had grown to cover more than just note-taking and journaling. I had come to depend on Evernote as the “Swiss Army knife” of my productivity tool kit. For example, I had used Evernote as my task manager, Evernote as a read-it later app like Pocket or Instapaper, and even Evernote as a sales and networking CRM. Evernote’s mission to “capture everything” had largely became how I used the tool.

Unfortunately, a few cracks started to appear with Evernote and my usage. First, my Evernote notes had become a bit of a monster, both conceptually and organizationally and in terms of the total number of notes. I felt a desire to to refine my note taking process and to slim down the number of notes I had. Second, Evernote as a product and company had seen better days.

The problems with Evernote as a company and as a product are not really the point of this post. But a quick summary of Evernote problems will often include: pricing changes, feature bloat, privacy around your notes, significant corporate changes, lack of product additions, and poor product performance (at least for me on Desktop).

Personally I rarely had much of an issue with the product or paying for a great product, like Evernote. But these concerns had built up over time and formed into on-going questions like: What’s going on with Evernote? Is it time to leave? How can I migrate? What should I migrate to?

A couple of months ago I finally decided to explore some Evernote alternatives and how I might migrate my notes. There are some solid Evernote replacements but I elected to switch to my notes to plain text files. Though Evernote’s corporate and product issues played a part in my decision too, my shift to plaintext files was less a rejection of Evernote, and more of a push to change up my way of organizing and working. To be clear: My goal was not to replace Evernote but to evolve my systems.

Migration is not an insignificant undertaking. Evernote makes your life easy for collecting, jotting ideas and then finding your old notes and documents later. If you have been a heavy user of Evernote, you likely have hundreds, if not thousands, of notes. Migrating to a new system is a time-consuming effort, and you still need to consider and adjust to your new way of working too.

There are several ways to migrate off of Evernote and onto another tool. One of the easiest note-taking tools to import into is Bear, a Mac/iOS markdown notes app. Lifehacker has a decent, though somewhat dated, post sharing several approaches for migrating to Microsoft’s OneNote, Apple Notes, or Simple Notes. Unfotunately none of these approaches work for migrating off of Evernote and onto plain text files. Even the best script, Ever2Simple, won’t keep your images, tags and meta-data when migrating to txt files. Losing so much information from my notes was a non-starter for me and forced me to find a new approach.

Fortunately, as I’ll show in this write-up, with a couple of steps and a combination of tools and scripts, you can effectively export your entire collection of notes out of Evernote and into markdown plaintext files. Most importantly, you can also still preserve the essentials of your old notes like images, tags, and even metadata like date created. Yoou can also maintain your legacy Evernote links between notes.

In this post, I’m going to show you how to migrate your notes out of Evernote and convert them into a collection of plaintext files in markdown. I’ll provide be providing a step-by-step guide to exporting out of Evernote and and processing into a format that you can open on any markdown editor. Additionally we will be sure to keep the images, links and meta for your original notes. Along the way, I’ll share some tips and my way of doing it too. At the end, I’ll conclude by briefly sharing a bit more about why I left Evernote and a few aspects of my new plain text life.

Let’s get started migrating our Evernote Notes!

The Power of Systematic Notes: A Book Review of How to Take Smart Notes by Sönke Ahrens

The first step in nearly “every intellectual endeavour” is to take a note. Writing notes is critical for how we learn, develop ideas and ultimately, create, and if you want to become a better writer or creative of any type, you need a better system and process for your notes.

Those ideas (take smart notes and build a connected, personal system of smart notes) are the central arguments of “How To Take Smart Notes” by Sönke Ahren, a book I recently read and have become somewhat obsessing over. Inspired by Niklas Luhmann (1927-1998), a well-known German social scientist and his method for managing his research and writing, Ahren explores how to be more productive, creative and organized using a system of deliberate note taking.

With over half of doctoral dissertations going unfinished (Lonka, 2003), Ahren’s main focus is on the organizational and creative problems of academic and nonfiction writers, and while the target audience is thesis writers, the lessons go well-beyond academia. I’d even argue that this provides one of the missing pieces to David Allen’s “Getting Things Done” method of productivity (Allen, 2001).

Throughout the text, Ahren argues for the importance of developing a special habit of note-taking and creating “smart notes.” Smart notes are a form of “learning through elaboration”, meaning we learn by putting complex ideas in our own words and by connecting them to other ideas. Smart notes are not just another way to collect stuff; their aim and goal is to foster and support creative and innovative output.

Based on these permanent, insight notes, we assemble a “knowledge management system” (my term) that he calls in German the Zettlekarten or in English the slip-box. It could also simply be called an archive. Ahren goes on to provide a tactical guide for developing and leveraging this interconnected knowledge system of smart notes throughout any creative project and ideally throughout life in general. Since smart notes form the nexus for what interests us, our organized thinking and on-going discussion questions, it’s both fodder for thought and where our writing should beginning.

Though technical and very specific at times, the book was a highly enjoyable read as Ahren journeys through processes underlying human learning, thinking, productivity and creativity. I highly recommend it for anyone interested who regularly writes (whether fiction or non-fiction) and for anyone who strives to better organize their knowledge and pursue innovation and creativity in any project.

Here are a few of the book’s key points that struck me in my reading and that I’m hoping to bring into my own learning and creative processes.

Towards a Science of Goals: Goal Setting as a Key Influence on Performance

“What you get by achieving your goals is not as important as what you become by achieving your goals.” – Michelangelo Buonarroti, Renaissance artist

In spite of our best intentions we fail at a lot of our goals. According to some estimates, a mere 8% of New Years’s Resolutions make it to the end of a year and nearly 80% have already failed or been abandoned by February. When it comes to academics and doctoral theses, over half will never be finished too (Lonka, 2003).

What causes such a high fail rate on our goals and can we do better?

While the idea of “s.m.a.r.t.” goals is often the first that comes to mind when you think about goals, there is an actual field of psychology dedicated to the science of goals. It’s well-researched and provides some very actionable approaches on how to better set and pursue our goals.

Started in the late 1960s, goal setting theory (GST) rests on its core claim that there is a relationship between goals and performance and that having a goal modifies how we behave. Though this idea might seem obvious now, this theory broke with the behaviorist tradition that interpreted much of how we behaved through either biological drives or rewards/punishments.

Goal Setting Theory attempt to explain how performance and motivation are affected by goals. One of the chief and earliest realizations of GST was the benefit to setting specific goals (over having no goals or vague “do your best” goals). It also found that there is a linear relationship between how difficult our goal is and how much better our performance is. To put it simply, the harder the goal, the higher the performance.

Subsequent research into goals has revealed many important aspects and key mechanisms on how to better plan and manage goals in companies, organizations and on a personal level. It has also crafted a powerful explanatory and actionable model called the High Performance Cycle (HPC).

In this post, we will be looking at the science of goals. While much of behavior is still viewed by many through the optics of biological drives or rewards/punishments, Goal Setting Theory (and other cognitive research on multiple goals like Goal Systems Theory) provide a much richer model for how goals function. This research indicates that two key things: 1. goals modify how we behave and 2. how we set goals affects aspects of how we perform and even how we feel.

As individuals and organizations, we can do better in pursuit of our objectives by learning the science of goals and applying its key lessons to how we set, track and manage our own goals!


NOTE 1: This is a fairly long and detailed post. If you want to skip the theory and just get to the applicable lessons, see the section below entitled, “How to Set Good Goals (according to science)”

NOTE 2: This post is the first part of an intended three-part series on goals and goal tracking. This first post focuses on the research related to goals, mostly goal setting theory. The second post in this series will look in more detail at three core aspects of goals: goal setting, goal tracking and goal management. The third and last post will look at a more practical, hands-on aspects to goal tracking, including how to build and leverage your own goal tracker tool in pursuit any objective you might have.

How to Create a Time Tracking Dashboard Using RescueTime, IFTTT and Google Sheets

RescueTime is one of my favorite ways to track my life. It’s a great passive way to know where your time is going on your computer. But how to collect your data and what to do with all that data once you get it?

Increasingly I’ve been using various automation services as one of my data collection methods. While you can use manual exports or code like QS Ledger to collect data from different tracking services, an automation service like IFTTT can automate the data collection. That way all of your data is stored into Google Sheets for easy access and even simple data visualization and data analysis.

Once your data is in Google Sheets, you can leverage custom functions and App Script to process and prepare that data. In turn, the ubiquity of Google Sheets means it’s easy to then pull data from there into your favorite visualization tools like Google Data Studio, Tableau or Plot.ly.

I see a lot of value in time tracking and time data. Personally, I started to track using time tracking tools when I become a freelance developer several years. Over time I discovered how time tracking made me conscious of my time usage, I learned to use time data in my weekly reviews and even explored a year of time tracking too.

In this post, I want to walkthrough setting up a simple time tracking dashboard with RescueTime, IFTTT and Google Sheets. First, we will use IFTTT for data collection from RescueTime into Google Sheets. We will then leverage some code in Google App Script to process and prepare our data. We will use some custom functions in Google Sheets to create some time dimensions from our date field. Finally, we will use some simple pivot tables and charts to do some personal data analysis. We are turning our tracking data into improved self-understanding.

The goal of this post show you some advanced functions for data processing inside Google Sheets. You’ll learn how to add and leverage custom code with Google App Scripts to extract information and do calculations. I’ll show you how to use array formulas to process columns of data in bulk. Finally, once we’ve done the hard technical work of data processing and data preparation, you’ll discover how easy it is to do some personal data analysis using pivot tables and charts and graphs.

Hold on to your spreadsheets. Let’s get started with some advanced data processing with Google Sheets!

How to Export, Parse and Explore Your Apple Health Data With Python

Most of us walk around carrying a small, sensor-infused computer. We call these devices “smartphones,” and they have more computing power and memory than the Apollo Space Capsules did when they went to the moon. Our phones contain sensors that detect movements, determine magnetic north, and even pinpoint us in relation to rotating satellites.

Our smartphones are incredible mini-trackers that can be used for both good and bad. On the good side, they can be used to help us know more about our health and behaviors. On the bad side, a lot of talk centers on privacy concerns, especially in relation to social media and internet usage but also go back to revelations about government surveillance and our smart phone data too. People seem worried about privacy and personal data, even though few know what data they actually have.

We should promote greater data protection and privacy, but we shouldn’t ignore the incredible opportunities we can gain from personal data too. So, while the bulk of the discussion these days is about personal data is on the negative’s, like data leaks and data privacy, I believe it’s a good time to try to understand the actual data we do have and how personal data and self-tracking might be used for self-improvement and even self-transformation.

For example, one of the most robust repositories about human health is on our smartphones, wearables and activity trackers. Leveraging a few sensors, our phones and wearables are able to interpret our movement patterns and tell us how many steps we took, how many stairs we climbed, how often we stood up, and many other activities. If you use a wearable with a Heart Rate Sensor, you can also capture your resting, active and sleeping heart rate and even know how long you slept too.

There are various ways and reasons why people track their lives, but when it comes to recording their daily movements, the most common method is with a wearable, activity tracker or smart watch. According to a Statista infographic, the most used wearables today are Fitbit, Apple Watch, Garmin, Mi-Band from XiaoMi, and Fossil. Interestingly, there are dozens of other devices with a much smaller marketshare but which offer an additional array of sensors to track other data points like blood pressure and HRV.

I recently created an open source project called Quantified Self Ledger. These are a collection of Python scripts that help to collect, process and aggregate data from various services like Fitbit, Apple Health, RescueTime and more. The initial goal is to collect and aggregate various self-tracking data. The end goal is to build a personal data dashboard and hopefully one day leverage it for more sophisticated data science and machine learning. In this post, I want to look at Apple Health. For example, how to export, parse and do some data analysis on your Apple Health data using Python. In later posts, we will look at a few other data points and tracking services.

If you are an Apple user, then your iPhone has been tracking your steps and a host of other health metrics. Some are directly recorded by the phone. Others are logged via other health apps that store their data into the Apple Health repository. If you also regularly wear an Apple Watch during the day, during workouts and at night, then you have even more data, like Heart Rate, VO2 Max, and possibility even Sleep.

In this post, we will be exploring Apple Health Data. First, we will look at some methods for exporting your Apple Health data, either using Apple’s raw export or an aggregated version using QS Access app. Second, we will then use some code to parse and process our raw Apple Health logs into more usable formats. Third, we will do some data exploration and data processing, so we can understand patterns and trends. Finally, we use this data to create some data visualizations in Python.

Whether you are merely curious or are trying to use tracking to support lifestyle changes and better habits, hopefully by the end of this post, you’ll understand what data you are collect and hope to start engaging with that data.

Why People Self-Track: Research on the Motivations Behind the Quantified Self and Self-Trackers

According research in 2016, sociologist Deborah Lupton estimates that there are well-over 160,000 tracking apps available in the app stores, including both for Android and Apple phones. This includes both explicitly tracking apps like Nomie and PhotoStats.io and various health and wellness apps like Strava and RunKeeper.

While we have yet to see a ubiquitous world of activity trackers, there are also dozens of wearables devices today like the Fitbit, Garmin, Jawbone UP, Nike+ Fuel, MiBand, and Apple Watch as well as dozens of other targeted devices and tools for quantifying your health and fitness.

Tracking and personal observation date back centuries. You can find strands of self-improvement through self-examination in both Ancient Greek and Ancient Chinese philosophers. Proceeded by the confessional writings of Saint Augustine of Hippo and Jean-Jacques Rousseau, the Victorian era was notable for the proliferation of personal diaries and journals, which allowed for a narrative format of self-reflection. Today’s digital age has not really changed the human quest, to borrow a phrase, to know thy self. We simply have more more tools and manners, both passive and active, to track our body, mind, time, environment or whatever. In short, it’s easier than ever to track a life.

Several centuries after Socrates declared the “unexamined life not worth living” is its digital equivalent, the “Quantified Self,” a neologism, a meetup, a movement and a life philosophy, whose tagline is “self-knowledge through numbers.” Considered one of the founders of QS, Gary Wolf is also one of the most active writers on the topic. His piece, The Data-Driven Life in the New York Times in 2010, captures the core of what self-trackers are pursuing as well as how diverse and divergent the QS movement is.

For example, one aspect is a technologically infused attempt at understanding human behavior. As he writes, “Ubiquitous self-tracking is a dream of engineers. For all their expertise at figuring out how things work, technical people are often painfully aware how much of human behavior is a mystery.”

As a journalist at Wired, Wolf has been chronicling the QS movement and its characters for nearly a decade. He subscribes to the idea that what today’s self-trackers are doing is not that different than what humans have been doing for centuries: personal observations. A few things have changed though. As he puts it:

Four things changed. First, electronic sensors got smaller and better. Second, people started carrying powerful computing devices, typically disguised as mobile phones. Third, social media made it seem normal to share everything. And fourth, we began to get an inkling of the rise of a global superintelligence known as the cloud.

Quantified self enthusiasts, self-trackers and just curious technologists can now leverage technology to deepen and widen their ability to observe and quantify themselves. But that still begs the question: Why do people track? Why Self-Tracking? Why pursue a quantified self?

In this post, I want to explore what motivates people to track their lives. Whether it’s a quantified self adherent or simply someone tracking their weight, health or fitness, a lot of people are tracking their lives today, and there hundreds of ways to do it. To help understand the space more, we will look the general categories tracking falls into. We will then look at a couple of research papers that attempt to survey and define the QS and self-tracking community. The goal of these papers is to understand what motivates someone to pursue self-tracking and create their self-tracking projects and experiments.

How to Track Your YouTube Watching (and Understand It)

How much time do you think you spend watching online videos like Youtube each day?

Be honest and take a guess.

While you probably have a pretty good sense of the type of content you are watching on YouTube, do you know how much time you spend watching?

You are likely underestimating the amount of time you spend watching online videos. According various studies, the average adult watches videos and TV for an hour longer than they estimate.

In short, we think we watch less TV and online videos than we actually do.

Personally, there have been nights where I’ll guiltily realize at 2am or later that I’ve just spent several hours streaming dozens of entertaining videos but wish I hadn’t. I’ll tell myself again and again “just one more video” as the dawn nears. I have oscillating feelings about YouTube, since it’s a free medium to watch amazing content, entertainment and educational.

If you have ever wondered what and how many videos you are watching online, you are in luck because you can transform Google’s YouTube Watch History into tracking statistics that tell you what you are watching and how much. Unlike other forms of tracking, it doesn’t require you to set anything up to access this tracking data.

The reality is that people spend a lot more time on YouTube: about an hour or more per day, according to Google. In comparison to my own tracking data, which I’ll go into detail on how you can get these stats too, I have watched well over 45 days worth of videos on YouTube in my life, and in the past two months, I’ve spent over a day and half each month watching YouTube videos or about 73 minutes per day.

While a lot of blogs and people online talk about protecting your privacy and about deleting your history on this or that service, I highly encourage you to collect your data before you delete it. Google and YouTube data is particularly interesting, since it can tell you a lot about your online behavior. You can see trends in what you watched during different times of your life. Collecting your Youtube watch history is also a great way to save as well as extend and augment your digital memory.

Our relation with technologies like smartphones, social media, apps, etc. is not always as simple as accept or reject. Instead, there can be a nuanced understanding and a conscious engagement. By tracking and analyzing our usage of certain technologies, we can better understand our engagement and make conscious choices about how and how much we we want use a technology like YouTube.

In this post, I want to share some ways to track and quantify your YouTube watching as a few steps you can use to change and improve your YouTube usage. We will look at three different techniques for tracking your Youtube video watching. Two are quite simple and require minimal setup but provide only a limited view of your YouTube History, while the third way will require some Python Code and Data Analysis in order to gain the most complete data on your Youtube History.

Hopefully this techniques can help you not only understand how much and what you watch YouTube, but change it too.

How I Write: My Favorite Tools and Apps for Writing

So, you want to write? And you’re looking for different tools to make your writing easier, better organized or adopted for a new publishing format?

I’ve published nearly 300 blogs and articles over the last several years, and, while the tools aren’t as important as the time, attention and process your put into writing, I’ve come to like and get pleasure out of certain writing software. These are the tools that put me into writing zen.

Whether you are working on the next great American novel, creating a work report, penning a poem, or just striving to get a shareable blog post published, here are my favorite tools for writing for writers.

A Matter of Fecal Matter: A 31-Day Scatological Self-Tracking Experiment

I tracked my poop for a month. Here is how I did it, what I did to track and process the and, in the end, what I learned from a matter of fecal matter.

First off, I did not literally touch or photograph my poop during this experiment. What I did do is log each and every time I passed a stool.

There are some few alternative ways to track your poop. Interestingly, there is a lot of talk in health and self-tracker space around the “gut microbiome” testing. This is where you analyze the bacterial makeup of your fecal matter, and there are several commercial companies that offer this service. The New York Times seems to love writing about it (here is a good article to get you started here). One of my favorite podcasts, The Quantified Body, has several in-depth episodes on the microbiome too. . It’s a topic that merits a separate experiment and discussion.

Similarly, in case you didn’t know it, there are already quite a few apps dedicated to tracking and logging your excrement. Apps like Poo Keeper, Poop Tracker and others, let you log and rate your poop using the Bristol Stool Scale (BSS). Developed over 20 years ago, this 7-type poop categorization system has become the gold standard for the clinical evaluation of your poo. This short medium post offers a great intro into the Bristol Stool Scale and poop tracking.

For this self-tracking experiment, I decided to keep things simple. I used a generic tracking tool called Hindsight to keep a log of my body waste over a 31-day period. Basically, each time I pooped, I took a few extra seconds to log the activity. Unfortunately I have yet to find a passive way to track my poop (yet).

While there are probably better ways to spend my time and plenty of other more “appropriate” experiments you can do to quantify your life or track your health, poop tracking provided an interesting and amusing opportunity to test out this new lifelogging tool, to practice my skills in data analysis and data visualization, and to learn a few more things about myself and my poop.

In this post, we will be looking at lifelogging and data visualization of fecal excrement over a month-long period, using Hindsight app.

Know Thy Blood: Common Questions and Answers About Blood and Blood Testing

Think of blood tests like a scoreboard.

On the one extreme, your lab results can help tell you and your doctor if you have a disease and they can give you early warning signs of future health problems. Blood testing get used throughout treatment to check on your responses and measure any side effects.

At the other extreme, biomarkers and blood tracking in general can indicate areas that are ok but not optimal. These tests can provide feedback on how to optimize towards improved wellness and longevity. If you are like me, then you can use your blood biomarkers to guide your health towards not just normal but optimal health.

There are a ton of reasons for regular blood testing. And, not surprisingly, blood testing has becoming a popular tool for self-trackers, biohackers, athletes and anyone striving for improved health in general. In my opinion, blood testing should be something everyone does regularly.

In this on-going series of posts on blood tracking and biomarkers, we are looking at how blood tests and other biomarker data can be use to to help self-trackers and people in general understand their health.

Unfortunately, blood testing, tracking and biomarkers can be quite confusing. Frankly most stuff in the medical space is rather intimidating. Almost intentionally so. There are hundreds of terms and a never ending range of opinions when it comes to our health. Fortunately, I think a basic and useful understanding of blood testing can be grasped relatively quickly, especially with the help of technology.

In this post, I want to answer many of the common questions about blood, blood testing and tracking, The first part focuses on questions like what is blood, what is a blood test, and how often to get your blood tested. In the conclusion, we will look at an initial answer to the most important question: Which blood tests you should get? Hopefully by the end of this post you should have a basic understanding of your blood and on how to get started with your blood tracking.

A WORD OF WARNING:

Dude, I’m not a doctor. If you aren’t sure, ask a real one. This post is not meant to be taken as professional medical advice. This is strictly my observations and opinions on what matters and doesn’t. While a lot of research and experimentation has gone into this, please seek professional medical advice along with your own personal research before doing anything stupid. Now on with the show.