Data and Metadata

Last updated:

2020-10-19 08:26

🚧 This document is in flux as Firn's API shifts and changes. 🚧

This document describes the data and metadata made available in Firn when it comes time to customize the rendering of your content in your layouts. This is a more advanced topic that may require a bit of comfort with Clojure.

Because Firn transforms org files into a data structure, it is possible to group content available to users who want more advanced control and flexibility with their data. The two main sections in the document provide the following:

Metadata - a description of the opinionated choices Firn makes to organize data for end-users.

Data - the raw data available when a file is parsed - and how to access it if you need more fine-grained control of your data.

Metadata

The following table describes all the data and functions "passed" to a layout each time a layout is applied to the data representing an org file. If the render function already does everything you need in your layouts, then this section may not be valuable for you.

Function/Data	Intent	Data-type
build-url	Function to more easily construct internal URLs	function
config	The site-wide config, containing everything!	map
date-created	The `#+DATE_CREATED` value of the processed file	string
date-updated	The `#+DATE_UPDATED` value of the processed file	string
file	The file as a data structure	map
file-links	A list of links per file	list
firn-tags	A map of tag names mapped to files.	map
firn-under	The `#+FIRN_UNDER` value of the file	string
logbook	A list of logbook entries of the processed-file.	list
logbook-total	The sum of all the logbook entries per file	string
meta	A map of metadata per file (logbook, links, keywords)	map
org-tags	A map of or org tags and associated headlines.	map
partials	a list of invokable partials from the `/partials` dir	list
render	See the Render Function document	function
site-author	The site author used for SEO.	string
site-desc	The site description used for RSS and SEO.	string
site-links	A list of all links across all documents	vector
site-logs	A list of all logbook entries	vector
site-map	A map of all files sorted by `#+FIRN_UNDER`	map
site-title	The site title used for RSS and SEO.	string
site-url	The site URL used to build link in templates.	string
title	The `#+TITLE` value of the file.	string

This may seem like a lot of information to make available to a layout template. And that's because it is! But thanks to destructuring in Clojure, you can make your templates only ask for what they need:

(defn project
  [{:keys [ render partials logbook] :as data}] ; < destructuring to make available only what you need.
  (let [{:keys [head nav main-section]} partials]
    (head)
    [:body
     (nav)
     [:main.main-container
      (page-header data)
      [:div.container
       (main-section render)
       (my-custom-logbook-chart logbook) ; passing logbook to a custom function, perhaps defined prior to the "project" function.
       ]]]))

The above template only needs access to render, partials and the logbook. Then functions, such as page-header (which must be defined above the project function) can simply take the data map and destructure what it needs again:

(defn page-header
  [{:keys [title logbook-total date-updated date-created firn-under]}]
  (let [rndr (fn [i s]
               (when i [:span.flex.pr2
                        [:h4.italic.bold.pr1 s " "]
                        [:h4.italic.thin i]]))]
    [:div.page_meta
     [:h1.page_meta_title title]
     [:div.flex
      (rndr date-created "Published: ")
      (rndr date-updated "Last Updated: ")
      (rndr firn-under "File Under: ")
      (when-not (= logbook-total "0:00")
        (rndr logbook-total "Time Logged: "))]]))

As per the table above, there is quite a bit of data that is made available to your layouts. If you want to know more about the shape of your data (such aslogbooks, org-tags, and so on), visit these pages to learn more.

Parsed Org File Data

Org files are parsed using a parser library called Orgize. When processing or serving your site, Firn sends your org-file as a string into Orgize, and gets back a data structure representing the contents of your file.

Currently, the raw parsed output of Orgize is stored in the file map under:as-edn. As per using your layouts as described in the Metadata section above, you can access the file map in your layouts.

A simple file with a few headings, tasks, logbook, etc, looks like this after being parsed to JSON and converted to EDN:

Click to view code sample

{:type "document",
 :children
 [{:type "section",
   :children
   [{:type "keyword",
     :key "TITLE",
     :value "Sample File!"}
    {:type "keyword",
     :key "FIRN_LAYOUT",
     :value "default"}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Meta",
     :properties
     {:file_under "Projects",
      :state "active",
      :date_completed "?",
      :links "?",
      :intent "Wiki",
      :date_started "<2020-03-01 Sun>",
      :slug "firn"},
     :children [{:type "text", :value "Meta"}]}
    {:type "section",
     :children
     [{:type "drawer",
       :name "LOGBOOK",
       :children
       [{:type "clock",
         :start {:year 2020, :month 3, :day 28, :dayname "Sat", :hour 15, :minute 45},
         :end {:year 2020, :month 3, :day 28, :dayname "Sat", :hour 18, :minute 29},
         :duration "2:44"}]}]}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Headlines <2020-03-27 Fri>",
     :properties {:foo "bar"},
     :children
     [{:type "text", :value "Headlines "}
      {:type "timestamp",
       :timestamp_type "active",
       :start
       {:year 2020,
        :month 3,
        :day 27,
        :dayname "Fri"}}]}
    {:type "section",
     :children
     [{:type "paragraph",
       :children
       [{:type "text",
         :value "and some stuff with a date: "}
        {:type "timestamp",
         :timestamp_type "active",
         :start
         {:year 2020,
          :month 3,
          :day 27,
          :dayname "Fri"}}
        {:type "text", :value "\r"}]}]}
    {:type "headline",
     :level 2,
     :children
     [{:type "title",
       :level 2,
       :keyword "TODO",
       :raw "Headline (2) with /keyword/",
       :children
       [{:type "text",
         :value "Headline (2) with "}
        {:type "italic",
         :children
         [{:type "text",
           :value "keyword"}]}]}]}
    {:type "headline",
     :level 2,
     :children
     [{:type "title",
       :level 2,
       :priority "B",
       :keyword "TODO",
       :raw "Headline 2 with priority",
       :children
       [{:type "text",
         :value "Headline 2 with priority"}]}]}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Some Links",
     :children
     [{:type "text", :value "Some Links"}]}
    {:type "section",
     :children
     [{:type "paragraph",
       :children
       [{:type "text", :value "A "}
        {:type "verbatim", :value "file:"}
        {:type "text", :value " link "}
        {:type "link",
         :path "file:file2.org",
         :desc "File 2"}
        {:type "text", :value "\r"}]}]}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Tables",
     :children
     [{:type "text", :value "Tables"}]}
    {:type "section",
     :children
     [{:type "paragraph",
       :children
       [{:type "text",
         :value
         "Some tables with texte markup in them\r"}]}
      {:type "table",
       :table_type "org",
       :tblfm nil,
       :children
       [{:type "table-row", :table_row_type "standard",
         :children
         [{:type "table-cell", :children [{:type "text", :value "1"}]}
          {:type "table-cell", :children [{:type "text", :value "2"}]}
          {:type "table-cell", :children [{:type "text", :value "3"}]}
          {:type "table-cell", :children [{:type "text", :value "4"}]}
          {:type "table-cell", :children [{:type "text", :value "5"}]}]}
        {:type "table-row",
         :table_row_type "rule"}
        {:type "table-row",
         :table_row_type "standard",
         :children
         [{:type "table-cell",
           :children
           [{:type "text", :value "foo"}]}
          {:type "table-cell",
           :children
           [{:type "verbatim", :value "foo"}]}
          {:type "table-cell",
           :children
           [{:type "italic",
             :children
             [{:type "text",
               :value "italic"}]}]}
          {:type "table-cell"}
          {:type "table-cell",
           :children
           [{:type "bold",
             :children
             [{:type "text",
               :value "bold"}]}]}]}]}]}]}]}

Interacting with datalimitation

As you can see, lots of data. Currently, Firn is not capable of interacting with this data very easily while you develop your Layouts. There are tentative plans to include a REPL, or at least the ability to println debug in future releases. For now, it is possible to independently use the Orgize parser online to see test results as JSON.