Data and Metadata

Last updated:
2020-09-23 18:43

🚧 This document is in flux as Firn's API shifts and changes. 🚧

This document describe the data and metadata made available in Firn when it comes time to customize the rendering of your content in your layouts. This is a more advanced topic that may require a bit of comfort with Clojure.

Because Firn transforms org files into a data structure, it is possible to group content available to users who want more advanced control and flexibility with their data. The two main sections in the document provide the following:

Metadata - a description of the opinionated choices Firn makes to organize data for end-users.

Data - the raw data available when a file is parsed - and how to access it if you need more fine grained control of your data.


Metadata

The following table describes all the data and functions "passed" to a layout each time a layout is applied to the data representing an org file. If the render function already does everything you need in your layouts, then this section may not be valuable for you.

Function/DataIntentData-type
configThe site wide config, containing everything!map
build-urlFunction to more easily construct internal urlsfunction
date-createdThe #+DATE_CREATED value of the processed filestring
date-updatedThe #+DATE_UPDATED value of the processed filestring
fileThe file as a data structuremap
file-linksA list of links per filelist
firn-underThe #+FIRN_UNDER value of the filestring
firn-tagsA map of tag names mapped to files.
logbookA list of logbook entries of the processed-file.list
logbook-totalThe sum of all the logbook entries per filestring
metaA map of metadata per file (logbook, links, keywords)map
org-tagsA map of or org tags and associated headlines.map
partialsa list of invokable partials from the /partials dirlist
renderSee the Render Function documentfunction
site-linksA list of all links across all documentsvector
site-logsA list of all logbook entriesvector
site-mapA map of all files as sorted by #+FIRN_UNDERmap
titleThe #+TITLE value of the file.string

This may seem like a lot of information to make available to a layout template. And that's because it is! But thanks to destructuring in Clojure, you can make your templates only ask for what they need:

(defn project
  [{:keys [ render partials logbook] :as data}] ; < destructuring to make available only what you need.
  (let [{:keys [head nav main-section]} partials]
    (head)
    [:body
     (nav)
     [:main.main-container
      (page-header data)
      [:div.container
       (main-section render)
       (my-custom-logbook-chart logbook) ; passing logbook to a custom function, perhaps defined prior to the "project" function.
       ]]]))

The above template only needs access to render, partials and the logbook. Then functions, such as page-header (which must be defined above the project function) can simply take the data map and destructure what it needs again:

(defn page-header
  [{:keys [title logbook-total date-updated date-created firn-under]}]
  (let [rndr (fn [i s]
               (when i [:span.flex.pr2
                        [:h4.italic.bold.pr1 s " "]
                        [:h4.italic.thin i]]))]
    [:div.page_meta
     [:h1.page_meta_title title]
     [:div.flex
      (rndr date-created "Published: ")
      (rndr date-updated "Last Updated: ")
      (rndr firn-under "File Under: ")
      (when-not (= logbook-total "0:00")
        (rndr logbook-total "Time Logged: "))]]))

As per the table above, there is quite a bit of data that is made available to your layouts. If you want to know more about the shape of your data (such aslogbooks, org-tags, and so on), visit these pages to learn more.

Parsed Org File Data

Org files are parsed using a parser library called Orgize. When processing or serving your site, Firn sends your org-file as a string into Orgize, and gets back a data structure representing the contents of your file.

Currently, the raw parsed output of Orgize is stored in in the file map under:as-edn. As per using your layouts as described in the Metadata section above, you can access the file map in your layouts.

A simple file with a few headings, tasks, logbook, etc, looks like this after being parsed to JSON and converted to EDN:

Click to view code sample

{:type "document",
 :children
 [{:type "section",
   :children
   [{:type "keyword",
     :key "TITLE",
     :value "Sample File!"}
    {:type "keyword",
     :key "FIRN_LAYOUT",
     :value "default"}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Meta",
     :properties
     {:file_under "Projects",
      :state "active",
      :date_completed "?",
      :links "?",
      :intent "Wiki",
      :date_started "<2020-03-01 Sun>",
      :slug "firn"},
     :children [{:type "text", :value "Meta"}]}
    {:type "section",
     :children
     [{:type "drawer",
       :name "LOGBOOK",
       :children
       [{:type "clock",
         :start {:year 2020, :month 3, :day 28, :dayname "Sat", :hour 15, :minute 45},
         :end {:year 2020, :month 3, :day 28, :dayname "Sat", :hour 18, :minute 29},
         :duration "2:44"}]}]}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Headlines <2020-03-27 Fri>",
     :properties {:foo "bar"},
     :children
     [{:type "text", :value "Headlines "}
      {:type "timestamp",
       :timestamp_type "active",
       :start
       {:year 2020,
        :month 3,
        :day 27,
        :dayname "Fri"}}]}
    {:type "section",
     :children
     [{:type "paragraph",
       :children
       [{:type "text",
         :value "and some stuff with a date: "}
        {:type "timestamp",
         :timestamp_type "active",
         :start
         {:year 2020,
          :month 3,
          :day 27,
          :dayname "Fri"}}
        {:type "text", :value "\r"}]}]}
    {:type "headline",
     :level 2,
     :children
     [{:type "title",
       :level 2,
       :keyword "TODO",
       :raw "Headline (2) with /keyword/",
       :children
       [{:type "text",
         :value "Headline (2) with "}
        {:type "italic",
         :children
         [{:type "text",
           :value "keyword"}]}]}]}
    {:type "headline",
     :level 2,
     :children
     [{:type "title",
       :level 2,
       :priority "B",
       :keyword "TODO",
       :raw "Headline 2 with priority",
       :children
       [{:type "text",
         :value "Headline 2 with priority"}]}]}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Some Links",
     :children
     [{:type "text", :value "Some Links"}]}
    {:type "section",
     :children
     [{:type "paragraph",
       :children
       [{:type "text", :value "A "}
        {:type "verbatim", :value "file:"}
        {:type "text", :value " link "}
        {:type "link",
         :path "file:file2.org",
         :desc "File 2"}
        {:type "text", :value "\r"}]}]}]}
  {:type "headline",
   :level 1,
   :children
   [{:type "title",
     :level 1,
     :raw "Tables",
     :children
     [{:type "text", :value "Tables"}]}
    {:type "section",
     :children
     [{:type "paragraph",
       :children
       [{:type "text",
         :value
         "Some tables with texte markup in them\r"}]}
      {:type "table",
       :table_type "org",
       :tblfm nil,
       :children
       [{:type "table-row", :table_row_type "standard",
         :children
         [{:type "table-cell", :children [{:type "text", :value "1"}]}
          {:type "table-cell", :children [{:type "text", :value "2"}]}
          {:type "table-cell", :children [{:type "text", :value "3"}]}
          {:type "table-cell", :children [{:type "text", :value "4"}]}
          {:type "table-cell", :children [{:type "text", :value "5"}]}]}
        {:type "table-row",
         :table_row_type "rule"}
        {:type "table-row",
         :table_row_type "standard",
         :children
         [{:type "table-cell",
           :children
           [{:type "text", :value "foo"}]}
          {:type "table-cell",
           :children
           [{:type "verbatim", :value "foo"}]}
          {:type "table-cell",
           :children
           [{:type "italic",
             :children
             [{:type "text",
               :value "italic"}]}]}
          {:type "table-cell"}
          {:type "table-cell",
           :children
           [{:type "bold",
             :children
             [{:type "text",
               :value "bold"}]}]}]}]}]}]}]}

Interacting with datalimitation

As you can see, lots of data. Currently, Firn is not capable of interacting with this data very easily while you develop your Layouts. There are tentative plans to include a repl, or at least the ability to println debug in future releases. For now, it is possible to independently use the Orgize parser online to see test results as JSON.