Readably Display Arbitrary Python Data

Learn a method for displaying nested python dicts and lists to the terminal

For a CLI program, or any program with text output, many programmers have been challenged with displaying data in the terminal in a readable format.

Either you are making a script to display data from a database via the python module sqlalchemy, or you are displaying the results of a RESTful API call via the python module requests, you likely have a handful of arbitrary complex python data structures to display. These might be a set of lists, dicts, ints, datetimes, and other objects.

Follow this guide and lets code together a way to display this data in a way you can easily read from the terminal.

In order to have data to display, I will use a complex data structure in the variable fake_data.

1
complex_data=(['hklmvKwrCboZpiwfUhoP', 'bass/explore/search/terms.php', 1678], {'party': 'utyUSpwEkRWicOcjDIeV', 'responsibility': 'williams/list/search.php', 'conference': 'TSrylvPlLNZifFJiqoFS'}, {'culture': {'apples': 'lcZORekRWSmDqfrQRPjc', 'oranges': ['hrobinsonhotmailcom', 6203562684090.42, 'tDNiboBzQtgWdFApUdYC'], 'pairs': {1: '2011-01-12 15:14:13', 2: 'DQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRG', 3: ['2000-1-10 09:05:01', 'parker-poole/blog/posts/category/login.php']}}})

One way of displaying text is using the popular library pprint which outputs in a format like this:

1
2
import pprint
pprint.pprint(complex_data)
(['hklmvKwrCboZpiwfUhoP', 'bass/explore/search/terms.php', 1678],
 {'conference': 'TSrylvPlLNZifFJiqoFS',
  'party': 'utyUSpwEkRWicOcjDIeV',
  'responsibility': 'williams/list/search.php'},
 {'culture': {'apples': 'lcZORekRWSmDqfrQRPjc',
              'oranges': ['hrobinsonhotmailcom',
                          6203562684090.42,
                          'tDNiboBzQtgWdFApUdYC'],
              'pairs': {1: '2011-01-12 15:14:13',
                        2: 'DQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRG',
                        3: ['2000-1-10 09:05:01',
                            'parker-poole/blog/posts/category/login.php']}}})

This is already a mostly readable way of displaying data, however this doesn’t work for user-facing output. All the python-specific syntax is left in the output, which end-users don’t care about. Also, the data isn’t all lined up which makes it slightly more difficult to read.

pprint just isn’t pretty enough. What we need is a way to display complex data in a neat and human-readable fashion.

Follow along as we code a function to display arbitrary data.

Defining the problem

In computer science, there is a concept called Dynamic Programming, where you solve a larger problem by solving the sub-problems first. In this case, the larger problem is displaying any arbitrary data, however the sub-problems are displaying units of data which python knows how to display through the built-in str conversion.

All python objects have a string representation through the __str__() function, so the smallest sub-problem to solve is to display any arbitrary string. After solving that problem, we can think about displaying lists of strings and dicts of strings. Even lists of dicts of strings.

Displaying a string

The first step to displaying any arbitrary data is to display a string. You may think it’s as easy as print(my_string), however that isn’t accounting whether the string is being displayed somewhere in the middle of the terminal window.

1
2
                                              We don't want the text to wra
p how we all dread

So how do we fix this?

Write a string-formatter function that knows where to wrap and how much to indent the next line.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def format_string(string, term_width=80, indent=0):
    # Use the string representation of any object to display
    string = str(string)
    # We can't format a string when there is no room left, so just return it
    if indent >= term_width - 5:
        return string
    # Format each line at a time
    formatted_lines = []
    for line in string.split('\n'):
        # Wrap lines that are too long
        while len(line) > term_width - indent:
            # get line up to width of terminal
            subline = line[:term_width - indent]
            # don't wrap in the middle of a word, if the words are small enough
            if ' ' in subline:
                subline = ' '.join(subline.split(' ')[:-1]) + ' '
            # append the line to the list of formatted lines
            formatted_lines.append(subline)
            # set-up next line to be formatted
            line = line[len(subline):]
        formatted_lines.append(line)
    # Since we aren't indenting the first line,
    if not formatted_lines:
        return ''
    # Return the lines with an indent on all lines except the first one
    return '\n'.join([formatted_lines[0]] +
                     [' '*indent + l for l in formatted_lines[1:]])

This function displays a string without any weird line-wrapping issues. Also, when displaying in the center of the terminal window, the next lines in the string are indented via the indent function argument.

The term_width argument is set to 80 to simulate a 80-character width terminal, however in a real terminal this can be obtained by reading the value of shutil.get_terminal_size()[0].

Now onto the fun stuff. With the function we made above, extra-long strings of text are formatted very nicely, like so:

1
2
3
short_string = 'a short string'

print(format_string(short_string))
a short string
1
2
3
very_long_string = 'a ' + 'very '*20 + 'long string for displaying the format_string\'s text-wrapping feature. As you can see the text does not wrap in the middle of a word.'

print(format_string(very_long_string))
a very very very very very very very very very very very very very very very
very very very very very long string for displaying the format_string's
text-wrapping feature. As you can see the text does not wrap in the middle of a
word.

Also, lets say we want to print a paragraph of text with a label on front. This is possible through format_string‘s indenting of every line except the first line.

An example of this is below.

1
2
3
label = 'the label for my text:'

print(label, format_string(very_long_string, indent=len(label) + 1))
the label for my text: a very very very very very very very very very very very
                       very very very very very very very very very long string
                       for displaying the format_string's text-wrapping
                       feature. As you can see the text does not wrap in the
                       middle of a word.

Displaying a list

Now that we can display arbitrary strings, now is time for displaying lists of strings that call the format_string function with the correct indent data.

We will work up to displaying a list of any arbitrary data, however for now lets simply display a list of strings.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def format_list(lst, term_width=80, indent=0):
    list_item_label = ' - '
    if not lst:
        return ''
    formatted_list = []
    indent_text = ''
    for item in lst:
        if isinstance(item, list):
            formatted_item = format_list(item, term_width,
                                         indent + len(list_item_label))
        else:
            formatted_item = format_string(item, term_width,
                                           indent + len(list_item_label))
        formatted_list.append(indent_text + list_item_label + formatted_item)
        indent_text = ' '*indent
    return '\n'.join(formatted_list)

This function returns the format_string representation of each item in the list, using ' - ' as the label for each item. For handling nested lists, format_list is called recursively when one of its elements is also a list.

Here are a couple examples of formatted lists:

1
print(format_list([1, 2, 3]))
- 1
- 2
- 3
1
print(format_list([short_string, very_long_string]))
- a short string
- a very very very very very very very very very very very very very very very
  very very very very very long string for displaying the format_string's
  text-wrapping feature. As you can see the text does not wrap in the middle
  of a word.

You can see that the text wrapping indents nicely for elements of the list. Lets see how the function does with nested lists.

1
2
3
list_of_lists = [['apples','oranges','pears'],'peaches',[['tangerines','bananas','avacados'],['carrots','oranges','beans']]]

print(format_list(list_of_lists))
-  - apples
   - oranges
   - pears
- peaches
-  -  - tangerines
      - bananas
      - avacados
   -  - carrots
      - oranges
      - beans

Now we’re talking. This nested list of lists of lists printed in a nice readable format to the screen.

1
print(format_list(complex_data))
-  - hklmvKwrCboZpiwfUhoP
   - bass/explore/search/terms.php
   - 1678
- {'party': 'utyUSpwEkRWicOcjDIeV', 'responsibility':
  'williams/list/search.php', 'conference': 'TSrylvPlLNZifFJiqoFS'}
- {'culture': {'apples': 'lcZORekRWSmDqfrQRPjc', 'oranges':
  ['hrobinsonhotmailcom', 6203562684090.42, 'tDNiboBzQtgWdFApUdYC'], 'pairs':
  {1: '2011-01-12 15:14:13', 2:
  'DQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSm
  JqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRG', 3: ['2000-1-10 09:05:01',
  'parker-poole/blog/posts/category/login.php']}}}

This function’s attempt at printing dicts in a list however epicly fails. To fix this problem, we need a dict-formatter function that is called when formatting an element of a list containnig a dictionary datatype.

Displaying a dict

Now that we have a list-formatting function, we need to format dict-datatypes as well.

Formatting the dict is a bit different than formatting a list. The main difference is that every item in the dict has a key as a label on front of each value. This means that we can’t shortcut the problem with something like format_list(dic.items()).

Lets use the same strategy of populating a list of lines to return, and then combining the lines into one string.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
def format_dict(dic, term_width=80, indent=0):
    max_key_len = max([len(str(k)) for k in dic.keys()])
    formatted_list = []
    indent_text = ''
    for k,v in dic.items():
        if isinstance(v, dict):
            formatted_line = format_dict(v, term_width,
                                         indent + max_key_len + 2)
        elif isinstance(v, list):
            formatted_line = format_list(v, term_width,
                                         indent + max_key_len + 2)
        else:
            formatted_line = format_string(v, term_width,
                                           indent + max_key_len + 2)
        formatted_list.append(indent_text +
                              '{}:'.format(k).ljust(max_key_len + 2) +
                              formatted_line)
        indent_text = ' '*indent
    return '\n'.join(formatted_list)

This simple function for displaying dicts sets up a list of lines to display by calling corresponding format functions recursively for each element in the dict.

1
2
3
nested_dict = {1: 'apples', 123: {'oranges': 20, 'tangerines': 30, 'carrots': 0}, 'something': list(range(3))}

print(format_dict(nested_dict))
1:         apples
123:       oranges:    20
           tangerines: 30
           carrots:    0
something:  - 0
            - 1
            - 2

There is still one problem. The format_list function doesn’t call format_dict when it finds a dict-type object. To fix this issue lets move on to the next step, which is combining all 3 functions.

Combining all 3 string-formatting functions to display any arbitrary data in one function

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def format_obj(obj, term_width=80, indent=0):
    # format a dict
    if isinstance(obj, dict):
        max_key_len = max([len(str(k)) for k in obj.keys()])
        return '\n'.join([(' '*indent if i > 0 else '') +
                          '{}:'.format(k).ljust(max_key_len + 2) +
                          format_obj(v, term_width,
                                     indent + max_key_len + 2)
                          for i,(k,v) in enumerate(obj.items())])
    # format a list
    elif isinstance(obj, list) or isinstance(obj, tuple):
        list_item_label = ' - '
        return '\n'.join([(' '*indent if i > 0 else '') +
                          list_item_label +
                          format_obj(item, term_width,
                                     indent + len(list_item_label))
                          for i,item in enumerate(obj)])
    # format anything else (with a __str__ representation)
    else:
        obj = str(obj)
        if indent >= term_width - 5:
            return obj
        formatted_lines = []
        for line in obj.split('\n'):
            while len(line) > term_width - indent:
                subline = line[:term_width - indent]
                if ' ' in subline:
                    subline = ' '.join(subline.split(' ')[:-1]) + ' '
                formatted_lines.append(subline)
                line = line[len(subline):]
            formatted_lines.append(line)
        return '\n'.join([(' '*indent if i > 0 else '') + line
                          for i,line in enumerate(formatted_lines)])
1
print(format_obj(complex_data))
-  - hklmvKwrCboZpiwfUhoP
   - bass/explore/search/terms.php
   - 1678
- party:          utyUSpwEkRWicOcjDIeV
  responsibility: williams/list/search.php
  conference:     TSrylvPlLNZifFJiqoFS
- culture: apples:  lcZORekRWSmDqfrQRPjc
           oranges:  - hrobinsonhotmailcom
                     - 6203562684090.42
                     - tDNiboBzQtgWdFApUdYC
           pairs:   1: 2011-01-12 15:14:13
                    2: DQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSm
                       JqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHolbVSmJqRGDQsaoDQPYHol
                       bVSmJqRG
                    3:  - 2000-1-10 09:05:01
                        - parker-poole/blog/posts/category/login.php

Now to display a random json file from the internet

1
2
3
4
import requests
import json
json_str = requests.get('https://my-json-server.typicode.com/typicode/demo/db').content.decode('utf-8')
dict_from_json = json.loads(json_str)
1
print(dict_from_json)
{'posts': [{'id': 1, 'title': 'Post 1'}, {'id': 2, 'title': 'Post 2'}, {'id': 3, 'title': 'Post 3'}], 'comments': [{'id': 1, 'body': 'some comment', 'postId': 1}, {'id': 2, 'body': 'some comment', 'postId': 1}], 'profile': {'name': 'typicode'}}
1
print(format_obj(dict_from_json))
posts:     - id:    1
             title: Post 1
           - id:    2
             title: Post 2
           - id:    3
             title: Post 3
comments:  - id:     1
             body:   some comment
             postId: 1
           - id:     2
             body:   some comment
             postId: 1
profile:  name: typicode

This neat representation of nested python lists and dicts is easy to read and flexible for displaying all types of data.

Feel free to use my code and apply it to your own projects.

Happy coding!

Get new posts by email:

Comments

Hint: To type code with Disqus, use 
<code><pre>insert.code.here()<pre/><code/>
▲