Notes for Nov-13

Today we will work with some text data that has been formatted into a tree structure (inspired by this online tool). I ran a simple script to break documents into sentences. Then, I fed each sentence into a tokenizer (to get individual words). Using the sentences, I built a tree structure. Every sentence starts at the same root node, branching off based on which word comes next. This creates a graph as sentences re-use the same starting words. We'll start with song lyrics for this example because they often have a repetitive structure that's easy to observe.

This is a general trend in text visualization -- while we can just show plain words, we often want to post-process the text to observe deeper structures that may not be obvious in a simple read-through of a larger document.

HTML for today:
Code for today: