A useful database

The generated tag-file (in json-format) holds all the needed information. But to use it in Python efficiently, we’ve got to do a few more things.

 

Get a valid json-file

The file tag.tg written by Ctags is not (yet) a valid json-file. You can easily verify this. Copy-paste the contents of the file into the editor on this page:
https://jsonlint.com/
Therefore we need to apply a few changes on it:

json_string = None
with open("tag.tg", "r", newline="\r\n") as f:
    json_string = f.read()
    json_string = json_string.replace("}\r\n", "},\r\n")
    json_string = "[" + json_string + "]"
    json_string = json_string.replace("},\r\n]", "}\r\n]")

with open("tag_validated.tg", "w", newline="\n") as f:
    f.write(json_string)

This code creates a new file tag_validated.tg, which is a valid json-file.

 

Create an sqlite database

Create a new database sqlite.db with the following code:

import sqlite3
import json

conn = sqlite3.connect("sqlite.db")
cursor = conn.cursor()
cursor.execute("""
    CREATE TABLE IF NOT EXISTS main
    (_type TEXT, name TEXT, path TEXT, pattern TEXT,
    typeref TEXT, kind TEXT, scope TEXT, scopeKind TEXT)
    """)

Now you’ve got an empty database file sqlite.db with a preformatted table called main. This table has columns _type, name, path, pattern, …

 

Fill the database

We’ve got an (empty) sqlite database and a valid json-file. Time to open the json-file and fill the database!

with open("tag_validated.tg", "r") as f:
    json_data = json.load(f)

for entry in json_data:
    cursor.execute("""
    INSERT INTO main
    (_type, name, path, pattern, typeref, kind, scope, scopeKind)
    VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
        (   entry["_type"] if "_type" in entry else None,
            entry["name"] if "name" in entry else None,
            entry["path"] if "path" in entry else None,
            entry["pattern"] if "pattern" in entry else None,
            entry["typeref"] if "typeref" in entry else None,
            entry["kind"] if "kind" in entry else None,
            entry["scope"] if "scope" in entry else None,
            entry["scopeKind"] if "scopeKind" in entry else None
        )
    )
conn.commit()
conn.close()

 

Put it all together

We’ll fit all this code neatly into a class named Ctags_parser:

import json
import sqlite3

class Ctags_parser():

    def __init__(self):
        self.json_string = None
        self.json_data = None
        self.conn = None
        self.cursor = None

        with open("tag.tg", "r", newline="\r\n") as f:
            self.json_string = f.read()
            self.json_string = self.json_string.replace("}\r\n", "},\r\n")
            self.json_string = "[" + self.json_string + "]"
            self.json_string = self.json_string.replace("},\r\n]", "}\r\n]")

        with open("tag_validated.tg", "w", newline="\n") as f:
            f.write(self.json_string)

        self.conn = sqlite3.connect("sqlite.db")
        self.cursor = self.conn.cursor()
        self.cursor.execute("""
            CREATE TABLE IF NOT EXISTS main
            (_type TEXT, name TEXT, path TEXT, pattern TEXT,
            typeref TEXT, kind TEXT, scope TEXT, scopeKind TEXT)
        """)

        with open("tag_validated.tg", "r") as f:
            self.json_data = json.load(f)

        for entry in self.json_data:
            self.cursor.execute("""
            INSERT INTO main
            (_type, name, path, pattern, typeref, kind, scope, scopeKind)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
                (   entry["_type"] if "_type" in entry else None,
                    entry["name"] if "name" in entry else None,
                    entry["path"] if "path" in entry else None,
                    entry["pattern"] if "pattern" in entry else None,
                    entry["typeref"] if "typeref" in entry else None,
                    entry["kind"] if "kind" in entry else None,
                    entry["scope"] if "scope" in entry else None,
                    entry["scopeKind"] if "scopeKind" in entry else None
                )
            )

    def name_exists(self, name):
        self.cursor.execute("SELECT name FROM main WHERE name = \"{n}\"".format(n=name))
        data = self.cursor.fetchone()
        if data is None:
            return False
        else:
            return True

    def __del__(self):
        self.conn.commit()
        self.conn.close()
        print("Database closed")

I’ve already added the function name_exists(name) to query for the existence of a name in the sqlite database. This function will come in handy soon.

 

Inspect the database

After running the code, you should get something like this:
 

 

You can download an awesome program to inspect the sqlite.db database:
http://sqlitebrowser.org/