Using Websockets with EmonPi MQTT Broker to Create a “Live Feed” Dashboard

This is not a complete description of the background tech; there is plenty of info on the web about websockets, mqtt, and the javascript libraries.

The motivating idea behind this experiment is to be able to have a live-updating dashboard with the minimum set of dependencies of the “install X” kind. Since the EmonPi aleady has a MQTT broker, it provides the basis for feeding data to the dashboard. Websockets allows a “web page” (it can be a file viewed in your web browser) to send and receive data, and update the page, using JavaScript. This is not particularly difficult, but there are several steps which took a couple of evenings to research and put into practice, so here are some notes (for myself in the future, and anyone else who finds it!).

If you are looking for some “homework reading”, a decent place to start is Steve’s Internet Guide.

Make Mosquitto Listen for Websockets Connections

Mosquitto is the MQTT broker on the EmonPi. It is configured to listen for connections which employ the “mqtt:” protocol. It is possible to add a websockets listener (“ws:” protocol), with the conventional port 9001 assignment as follows:

  • SSH onto the EmonPi and navigate to /etc/mosquitto.
  • Modify the mosquitto.conf file to add “listener 9001” and “protocol websockets”, see below. I have also added explicit lines for the default mqtt listener on port 1883 in the interest of clarity, although I believe they are not required. You will need to use “sudo”. Alternatively, it should be possible to add a file to the “include_dir”.
  • Restart mosquitto using “sudo systemctl restart mosquitto.service”
pid_file /var/run/mosquitto.pid
persistence false
persistence_location /var/lib/mosquitto/
log_dest file /var/log/mosquitto/mosquitto.log
include_dir /etc/mosquitto/conf.d

listener 1883
protocol mqtt

listener 9001
protocol websockets

allow_anonymous false
password_file /etc/mosquitto/passwd
log_type error

It should now be possible to test two things: that the existing mqtt protocol listener, which is relied on to service data to EmonCMS, and that the websockets listener us “up”. I used “MQTT Explorer”, a free and simple client, which should be set up to not validate a certificate and to have an empty “Basepath” (it defaults to “ws”).

Create the Websockets Dashboard with HTML and JavaScript

My primary aim for this experiment is to be able to co-opt EmonPi to broker air quality data from a home-brew particulate matter, VOC, NOx, CO2 sensor combo, but I’m using the existing emon data to demonstrate the concept, which boils down to “guages” using the Google Charts toolkit, and scrolling line charts using Chart.js.

Here is the code to hack about with, based on snippets from various places, with modifications and updating to a recent. It should just live in a plain text file with a “.html” extension, and can be opened in your web browser. It is not beautiful but demonstrates the concept. There is some logging to the “console”, which is where error messages will also appear. Hit F12 on Firefox or Edge (or Chrome too I think) to find the console.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <!-- This helps with viewing on mobile devices -->
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <title>Real-Time Charts</title>
    <!-- Google charts -->
    <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
    <!-- Chart.js -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/3.9.1/chart.min.js" integrity="sha512-ElRFoEQdI5Ht6kZvyzXhYG9NqjtkmlkfYk0wr6wHxU9JEHakS7UJZNeml5ALk+8IKlU6jDgMabC3vkumRokgJA==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <!-- The Paho Javascript for MQTT over Websockets -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.min.js" type="text/javascript"></script>
    <!-- Bootstrap CSS - can be removed but will help with the styling -->
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/css/bootstrap.min.css" integrity="sha512-SbiR/eusphKoMVVXysTKG/7VseWii+Y3FdHrt0EpKgpToZeemhqHeZeLWLhJutz/2ut2Vw1uQEj2MbRF+TVBUA==" crossorigin="anonymous" referrerpolicy="no-referrer" />
    <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='main.css') }}">
</head>
<body>
<script type="text/javascript">
    // load the google charts stuff and only after it is loaded should we start to set up the data acquisition
    // otherwise, we find the charts library is being called upon before it exists!
    google.charts.load('current', {'packages':['gauge']});
    google.charts.setOnLoadCallback(doWhenReady);

    function doWhenReady() {
        // Create a client instance. NB clientid SHOULD BE DIFFERENT between browser clients; the following should work fine in a home environment.
        var clientId = "client-" + Date.now().toString();
        client = new Paho.MQTT.Client("192.168.1.1", 9001, clientId);
        // set callback handlers
        client.onConnectionLost = onConnectionLost;
        client.onMessageArrived = onMessageArrived;
        // connect the client
        client.connect({onSuccess:onConnect, userName: "emonpi", password: "emonpimqtt2016"});

        // called when the client connects
        function onConnect() {
          // Once a connection has been made, make subscriptions
          console.log("onConnect");
          client.subscribe("emon/emonpi/vrms");
          client.subscribe("emon/emonpi/power1");
        }

        // called when the client loses its connection
        function onConnectionLost(responseObject) {
          if (responseObject.errorCode !== 0) {
            console.log("onConnectionLost: " + responseObject.errorMessage);
            document.getElementById("status").innerText = "Lost connection due to: " + responseObject.errorMessage;
          }
        }

        // called when a message arrives. Note that the full topic name is aka "destinationName"
        function onMessageArrived(message) {
          document.getElementById("status").innerHTML = "";
          console.log("onMessageArrived: " + message.destinationName + " : " + message.payloadString);
          if (message.destinationName == "emon/emonpi/vrms") {
            // "+" converts string to numeric, which is then rounded by toFixed(), which returns a string!
            var vrms = (+message.payloadString).toFixed(2);
            // Google chart guage
            vrmsGaugeData.setValue(0, 1, +vrms);
            vrms_gauge.draw(vrmsGaugeData, vrmsOptions);
            // scrolling line chart.js
            if (vrmsChartData.data.labels.length === 20) {
                vrmsChartData.data.labels.shift();
                vrmsChartData.data.datasets[0].data.shift();
            }
            // we don't have the time in the mqtt message!
            var timestamp = new Date().toLocaleTimeString();
            vrmsChartData.data.labels.push(timestamp);
            vrmsChartData.data.datasets[0].data.push(vrms);
            lineChartVrms.update();
          } else if (message.destinationName == "emon/emonpi/power1") {
            var power1 = message.payloadString;
            // Google
            power1GaugeData.setValue(0, 1, +power1);
            power1_gauge.draw(power1GaugeData, power1Options);
          }
        }

        // ------------- Google Charts -----------
        var vrmsGaugeData = new google.visualization.arrayToDataTable([
            ['Label', 'Value'],
            ['Vrms', 240]
        ]);
        var power1GaugeData = google.visualization.arrayToDataTable([
            ['Label', 'Value'],
            ['Power', 0]
        ]);
        var vrmsOptions = {
            min        : 200,
            max        : 260,
            minorTicks : 5,
            greenFrom  : 230,
            greenTo    : 250,
            majorTicks : ['200', '210', '220', '230', '240', '250', '260']
        };
        var power1Options = {
            min        : 0,
            max        : 5000,
            minorTicks : 5,
            majorTicks : ['0', '1kW', '2kW', '3kW', '4kW', '5kW']
        };

        var vrms_gauge = new google.visualization.Gauge(document.getElementById('vrms-gauge'));
        var power1_gauge = new google.visualization.Gauge(document.getElementById('power1-gauge'));
        vrms_gauge.draw(vrmsGaugeData, vrmsOptions);
        power1_gauge.draw(power1GaugeData, power1Options);

        // chart.js
        var chartOptions = {
            responsive: true,
            tooltips: {
                mode: 'index',
                intersect: false,
            },
            hover: {
                mode: 'nearest',
                intersect: true
            },
            scales: {
                xAxes: {
                    display: true,
                    scaleLabel: {
                        display: true,
                        labelString: 'Time'
                    }
                },
                yAxes: {
                    display: true,
                    scaleLabel: {
                        display: true,
                        labelString: 'Value'
                    }
                }
            }
        };

        // this allows us to have common chart options, while varying the scale in each chart
        var vrmsChartOptions = structuredClone(chartOptions);
        // use sensible range, which will be expanded if the data goes outside "suggested"
        vrmsChartOptions.scales.yAxes.suggestedMin = 230;
        vrmsChartOptions.scales.yAxes.suggestedMax = 250;

        var vrmsChartData = {
        type : 'line',
        data : {
            labels: [],
            datasets : [{
            label : 'Vrms',
            backgroundColor : 'rgba(255, 136, 0, 0.5)',
            borderColor : 'rgba(255, 136, 0, 1.0)',
            fill: true,
            data : []
            }]
        },
        options : vrmsChartOptions
        };

        const lineChartVrms = new Chart('vrmsChart', vrmsChartData);
    }
</script>

<h1>EmonPi Live Feed</h1>
<div class="container">
    <div class="row">
        <div id="vrms-gauge" style="width: 120px; height: 120px;"></div>
        <div id="power1-gauge" style="width: 120px; height: 120px;"></div>
    </div>
    <div class="row" id="status">loading data...</div>
    <div class="col-10">
    <div class="card">
        <div class="card-body">
            <canvas id="vrmsChart"></canvas>
        </div>
    </div>
</div>
</div>
</body>
</html>

Don’t forget to change the IP address for the MQTT broker!

Serving the Dashboard

While the above HTML + JavaScript works fine as a file on your PC (so long as you have a network connection to acquire all those JavaScript libraries from), it can also be placed on the EmonPi.

I have opted to create a separate area from EmonCMS in the interest of avoiding too much risk of cock-up, future muddle, etc… The files in this separate area will all be “static” in the sense that they are just served to users as they are (no PHP etc). The dashboard page above IS static in this sense, since the JavaScript executes on your PC inside the web browser; all the EmonPi does is send it.

EmonCMS uses the Apache2 web server, so it is easy to make it listen on a different port (EmonCMS uses port 80, and I have chosen port 8001) using the following commands (in a SSH).

Create the place where the HTML file will live

cd /var/www
mkdir static

Check the directory ownership is correct using “ls -la”, which should show “drwxr-xr-x 2 pi pi 4096 Nov 27 23:02 static”.

If necessary “chown pi:pi static”.

Place the HTML + JavaScript file here (I suggest using the scp command, for example “scp dashboard.html pi@192.168.1.1:/var/www/static”).

Change the Apache2 config

There are two things to change, first to make apache2 listen on port 8001, and secondly to tell it to use the files in /var/www/static in connection with port 8001 requests.

cd /etc/apache2
sudo nano ports.conf

Add a single line “Listen 8001”, then save and exit.

For the second step, enter the “sites-enabled” directory. I chose to copy the existing emoncms.conf file, naming the copy “static.conf” and editing it to contain:

<VirtualHost *:8001>
  ServerName localhost
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/static

  # Virtual Host specific error log
  ErrorLog /var/log/static/apache2-error.log

  <Directory /var/www/static>
    Options FollowSymLinks Indexes
    AllowOverride All
    DirectoryIndex index.html
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

The changes are fairly obvious, but note that I have also added “Indexes” against “Options”. This makes apache give a listing of all the files in “static” when I use the URL “http://192.168.1.1:8001”. If you only have one dashboard, then simply name it “index.html” and it will appear when that URL is used.

Make a directory for logs

Otherwise apache2 will not restart.

cd /var/log
mkdir static

Restart Apache2

To load the new config.

sudo systemctl restart apache2

Make sure EmonCMS is still working, then visit your new site! It should work adequately on a mobile phone, once rotated to landscape format.

 

Using MQTT in Open Energy Monitor to Capture External Device Data

I struggled to find information about this using web searches, so here is a condensed “how to”. The scenario I have is using a home-brew ESP8266 based device attached to my solar PV inverter which I want to relay definitive power output to my Open Energy Monitor via MQTT. This seems quite simple in principle, and is simple in practice, but seemingly not well documented.

First thing is to publish the data to an MQTT broker on the emonpi with a topic which starts “emon/{source}/{key}”, replacing {source} with a suitable name for the data source and {key} with the attribute name for the data being sent. In my case, I used “emon/solis/power” as the data source is the output power for my Solis PV inverter. The message payload is simply the data value. This immediately makes the published data appear on the “Inputs” screen of EmonCMS.

Two refinements/possibilities:

Send JSON

Rather than just a single value, send several in the same message. There are two ways to do this:

a) use a topic of form “emon/{source}”

b) use a topic of form “emon/{source}/{key}”

If the message payload sent to the MQTT topic “emon/solis/power” looks like {“ac”: 90, “dc”: 120}, option (a) creates EmonCMS inputs “ac” and “dc” under “solis”, while option (b) creates inputs called “power_ac” and “power_dc”.

Include a timestamp

To do this, simply include an extra field in the JSON called “time”, with a value which is the Unix time. If you are testing, the Unix time needs to be fairly close to the actual time (ignoring summer time) otherwise EmonCMS will indicate “inactive”, but it still captures the data.


Aside: I used the VSMqtt plugin for VSCode as the MQTT client, as I’m using the PlatformIO plugin to develop my ESP8266 code (using the Arduino libraries).

Safe Use of GPIO on ESP8266 NodeMCU Boards

It is well known that, unlike a Raspberry Pi or old-school Arduino, the ESP8266 has some quirks which make using the broken-out pins D0-D8 non-trivial. Google/Bing/etc will easily confirm this. I have, however found lots of long winded, unclear, and conflicting information. This post is my aide memoir, shared in case anyone else might find it useful. The hardware platform is “HW-628 V1.1”, from one of the many cheap Chinese sellers on Alibaba. I’m using the Arduino libraries with PlatformIO (within VSCode), but would expect the same results using the Arduino IDE.

My baseline assumption is that inputs and outputs will normally be used in “active low” mode, with INPUT_PULLUP used as the pinMode for inputs. There are two problems which will easily be found without care: 1) [as outputs] some pins are driven low on boot, usually with quite a few pulses, which will cause an active low relay to stutter; 2) [as inputs] holding some pins low will block the booting.

Tests for the effect of booting (and flashing) on output states were undertaken using a logic analyser (an Open Bench Logic Sniffer with OLS software) with a weak pull-up on the pin under test. I am using the D0-D8 notation as marked on the NodeMCU board. The Arduino library pin numbers differ, but the mapping is widely published.

The following pins were found to be OK to use without any restrictions, as inputs or outputs: D0, D1, D2, D5, D6, D7. The only caveat for my hardware is that D0 is connected to one of the on-board LEDs (active low).

The following CANNOT be used as outputs (with the exception of LED signalling*) since they are affected on boot (and when flashing): D3, D4, D8. D8 was found to hold a low value during flashing and during and slightly after the reset pulse. D3 and D4 showed a burst of pulses. You might get away with using D8 in its “TX2” UART alter ego since the flash/boot glitch wont make a valid data frame. [* – the NodeMCU Devkit board I have does have an LED on D4 = GPIO2 and this can be used for signalling from programmes].

D3, D4, and D8 can be used as inputs but only with some care:

  • D3 and D4 can be used as active low inputs only AFTER the boot sequence is completed. Push-switch inputs would be safe. Anything which would pull these pins low on boot will block it. D4 is connected to the on-board LED which is marked “COM” (and is active low).
  • D8 can be used as an active high input only AFTER the boot sequence. Push-switch inputs would be safe. I would use this last as I don’t like circuits to contain a mix of active low and active high inputs; logic should normally be consistent to reduce bug risk.
  • Since these GPIOs have output functions during boot and flashing, if there is a chance that a push switch input would be operated during those states, the switch should connect to ground or supply via a current limiting resistor.

An aside: when setting pins as outputs with pinMode, it is worth setting the output value to HIGH with digitalWrite() BEFORE changing the mode from its default as an input. Otherwise, assuming active low working again, you will be likely to see an output glitch due to a default low state pertaining when the pinMode() is applied.

Digital Elevation Models & Mining History

Digital elevation models are expressions of the surface of a planet as data. These, and the software to view or process the data, have become freely available over recent years. The software ranges from easily usable online viewers to PC-based tools requiring intermediate levels of IT skill. This all makes for an interesting and useful resource for people interested in mining history, whether as a casual interest or as a more focused amateur historian. This article seeks to provide an introduction to the topic and to illustrate what is possible. A shortened version is being published in the Peak District Mines Historical Society (PDMHS) members’ newsletter. If you find this article interesting, you may be interested in joining and participating in membership activities, but we also have public activities when pandemics allow.

I am avoiding technical detail, and not providing a “how to” guide; there is an abundance of information available on the web which should suffice, although a mining history focused “how to” guide may be created for PDMHS members if there is interest.

A Brief Introduction to Digital Elevation Models (and Lidar)

I am using “digital elevation model” (hereafter: DEM) as a generic term as well as two more specific terms: “digital terrain models” (DTM) and “digital surface models” (DSM). A DSMs give the surface with things on it – trees, buildings, walls, etc – whereas DTMs, in UK parlance, show the ground level (in the USA, a DTM is augmented with information about surface features such as rivers). Someone with an interest in forest canopies would normally be interested in a DSM, whereas we mining history people are much more likely to be interested in what is beneath the vegetation, hence in DTMs.

DEMs are usually created using the data from flying aircraft, with a considerable amount of sophisticated data processing to get from the raw data to usable DEM data. Radar has been used, although Lidar (an acronym for “light detection and ranging”) is the most likely source of DEMs which we will use. Lidar and radar use similar techniques: bouncing light or radio waves off objects and measuring the pulses which arrive at the sensor to “see” objects. Lidar equipment is extremely expensive. A cheaper alternative is to use photogrammetry, which entails taking numerous overlapping photographs from an aircraft, and using software to work out what the surface elevations must be to cause the observed changes between adjacent photographs. Photogrammetry can be achieved from professional grade unmanned aerial vehicles (aka “drones”). See, for example the 3D model of Magpie Mine, with online viewer, created by Peak Drone Imaging. Some specialist providers also fly Lidar drones.

What are DEMs Useful For?

Before listing what we can use DEMs for, it is fair to mention that even the higher-resolution images are no substitute for an experienced archaeologist on the ground. Such a person will see things, feel them under their feet, draw inferences from changes in vegetation, and observe traces of mineral, etc. Unfortunately such people are in short supply. Aerial photographs may also be more revealing than images created from DTMs, especially high resolution black and white images, but also when the character of vegetation is shown, although the best resolution of freely available aerial photographs is generally not good enough to compete with the latest Lidar-based DTMs.

Fieldwork (preparation) from your desk chair: before venturing out, 2m or higher resolution DTMs, can be a useful way of “seeing” what is there. This kind of prospecting can be really useful when the terrain is difficult. Alternatively, there may be places of interest on private property with no access.

Seeing though undergrowth is a major advantage of DTMs, making visible what would otherwise remain fairly well hidden.

Validating grid references and sketch maps; there are abundant imprecise locations from before the days of GPS, when sometimes an approximate 100m grid reference was the best that could be achieved. This also applies to British Geological Survey maps, which are often based on very old surveys and locate veins quite inaccurately.

Visualising the 3D character of the landscape is one nice application, and one which works even with 50m spatial resolution. This can be either quite subtle shading on a flat map, to given an impression of shadow, or a more virtual-reality-like presentation. Contour lines are really in this category too.

What Data is Available?

There are several good sources of freely-downloadable data (but with some licence terms):

Ordnance Survey OS Terrain 50 is a DTM with a spatial resolution of 50m. It is not available as GeoTiff (see below), but in alternative grid-based formats and as contours.

NASA mapped almost the entire land surface of planet Earth in the Shuttle Radar Tomography Mission (SRTM) at 1 arc second spatial resolution, which is approximately 30m. You can browse maps online and download the data using the USGS Earth Explorer, and there is a SRTM downloader plugin for GQIS (see below).

The Environment Agency has surveyed many parts of England over the last 20 years, at a range of spatial resolutions between 2m and 0.25m. This work was particularly driven by a desire to model and predict flooding, so the smallest resolution surveys closely follow draining networks near large urban/suburban areas such as Sheffield. Since 2016 they have been undertaking the National Lidar Programme, which will complete its survey of the whole of England at 1m spatial resolution in 2021/2 (Lidar surveys are taken during the winter months while vegatation is largely dormant). The vertical accuracy is claimed to be below 0.15m and both DTM and DSM datasets are available. This is all available under an Open Government Licence. This is outstanding!

While the OS and NASA datasets are useful for visualising the physical geography, a spatial resolution of 1m or less is essential for picking up mining historical features other than large scale open-casting. Consequently, most of the rest of this article will look at the Environment Agency data, in particular the 1m National Lidar Programme data as it is available in nice square blocks, whereas the 0.5m and 0.25m surveys are quite limited in coverage and tightly follow streams and rivers.

Visualising DEMs

The simplest way of representing a DEM is as a square grid of altitudes, and this is the most likely way in which data is provided. The grid spacing, known as the spatial resolution, typically varies from around 100m down to 0.5m, or sometimes as small as 0.25m. A grid of elevations is also quite easy for computer programmes to process. The most favoured current format is GeoTiff, which is an extension of the Tag Image File Format (Tiff) to include data about the geospatial location which the data relates to. The simplest way of rendering a DEM for viewing is to have the elevation values correspond with different brightness values, from black (lowest elevation) to white (highest elevation), so GeoTiff is a neat solution. A bit of desk research will turn up lots of alternative file formats. Fortunately, most geospatial software can handle several of them.

Unfortunately, just opening one of these GeoTiff files on your PC will usually just give you what looks like a plain white image, but with the right software we can get something looking a little surreal or medical.
(All images are linked to a larger version)
We can do a bit better by creating a pseudo-colour image, where the elevations are mapped to a colour. The colour scheme can be semi-naturalistic like a traditional atlas, a simple grading from one colour to another via white, a rainbow or a garish scheme designed to highlight a particular elevation range. One really effective technique is to generate “hill shading” by  simulating light and shadow from an imaginary sun. If you visit the OS Terrain 50 web page, there is an interactive map where you can adjust the azimuth to alter the hill shading. Hillshade works well when overlaying a base map or aerial photography.

Grey-scale representation of a DTM. Monsal Head is at the top and Lathkill Dale at the bottom

Wye to Lathkill Pseudocolour DTM

Wye to Lathkill Hillshade Overlaying OpenStreetMap

Hill shading turns out to be quite effective at picking out even quite slight surface features with the higher resolution DEM data. When hill shading is used to augment a map, steep slopes can appear very dark, so people often use multi-directional hill shading, with three or more imaginary suns all shining at the same time. This sounds very unrealistic, but the images are pleasantly usable.

Some other ways of representing a DEM are as contours, which are not so easy to work with “as data” but super if you are map-making, and as the surfaces of 3D objects, which would be useful for creating photo-realistic images and “virtual reality” environments. This article will not look at 3D visualisations.

Visualising DEMs on the Web

My top recommendation is to use the Environment Agency Geomatics Team’s Lidar Composite viewer, but there are other services to explore via their Open Data Products page. This is a very interesting resource which includes clear information about Lidar, the accuracy of the data, and the way raw data is turned into a DTM or DSM. At the time of writing, the latest data which is included is the “2020 Composite”, but this will change as the National Lidar Programme progresses (link to catalogue of all completed and planned surveys).

Environment Agency Lidar Composite Viewer, with key controls annotated: Map layer, Base map, Spatial Resolution, and Placename Search.

At this point, I urge you to go and explore! The most important controls are the spatial resolution selection and the map layer. Once the Map layer control is opened, click on the little eye symbols to switch different effects on and off, or on the three dots to change the opacity of the layer. Experiment with hill shading, pseudo-colour, overlaying hill shade onto base maps (you will have to decrease the opacity of the hill shade), and looking at slope and aspect. The area shown in the image, above, is Grin Low, above Poole’s Cavern in Buxton, providing a dramatic illustration of the legacy of lime burning which is only partially evident as one walks around the woodland. The Thatch Marsh and Burbage Colliery area, not far to the West of Grin Low shows the causeways and pit locations quite clearly, as well as the pack horse hollow-way leading from the pits towards the lime kilns where much of the coal was consumed. The area and its history is thoroughly described in “Coal Mining near Buxton: Thatch Marsh, Orchard Common and Goyt’s Moss” by John Barnatt, in Mining History 19-2 (not currently online) which has several maps drawn from proper fieldwork, which make for an instructive comparison with Lidar armchair exploration.

An alternative viewer is the Environment Agency Survey Open Data Index Catalogue. This gives access to more datasets, including DSMs (although only in the 2017 Composite, not the 2019 version), but I don’t find it quite as pleasant to use.

For the More Adventurous – QGIS

QGIS (formerly Quantum GIS) is open source software with professional-level geographical information system (GIS) capabilities. Software with such power comes with challenges, and I would only recommend people who are confident IT users even taking a look. I hope those who are confident will be able to repeat the following examples, and are maybe motivated to learn more. There are a lot of resources explaining how to use QGIS on the web; just be aware that the latest version is QGIS 3, and many resources refer to the previous release. If you do download QGIS, choose the latest “long term release” version. I am going to use some more technical terms in this section, and will not explain them all, expecting readers can do their own web searches.

Before doing anything else, I recommend adding OpenStreetMap as a base map layer, if only so you know where you are! This is achieved by adding an “XYZ Tiles layer” (see this guide). The URL to add is “http://a.tile.openstreetmap.org/{z}/{x}/{y}.png” (without the quote marks, but with those curley brackets). There are lots of XYZ servers available – try a google search – but the Bing Maps satellite imagery service is a good companion to the OpenStreetMap: “http://ecn.t3.tiles.virtualearth.net/tiles/a{q}.jpeg?g=1“.

The starting point for all which follows is QGIS with a map in view and two panels – headed “Browser” and “Layers” – on the left hand side. If these panels are not visible, use menu View > Panels.

Getting Data for QGIS

I am only going to consider using Environment Agency data, for which there are two ways of using the DEMs in QGIS: downloading GeoTiff files, and accessing a WMS map tile server. Once set up, the WMS approach is quicker and easier but you get less control over how the DEM is rendered. Consequently, I suggest using the Defra Survey Data Download service. The workflow for using this site is generally: draw your area of interest on the map, click “Get Available Tiles”, then choose which dataset you want. Ignore the shapefile upload option; the icons for drawing your area are just beneath the upload grey box. This catalogue covers all the published datasets, with downloads being 5km x 5km squares. At 1m spatial resolution, that amounts to 5000 x 5000 = 25 million data points, so even these tiles are quite large files.

Once downloaded, you can just drag-and-drop the file into QGIS. If you do this, you will see that QGIS chooses a grey scale for each based on black = the lowest elevation and white = the highest elevation. This makes things look blocky if you load several 5km square tiles since each has a different min/max elevation. This can be fixed quite easily, but doesn’t matter for some uses (see below). I generally work with the OpenStreetMap and satellite imagery as the bottom layers and put DEM layers above (and the notes below assume this).

Playing with Hill Shading, Pseudo Colouring, and Overlaying Maps

For this section, we will work within a single 5km tile. I chose the SK16NE DTM from the 1m National Lidar Programme (aka DTM_1565) to work through what follows, which has Magpie Mine almost at the centre. The Magpie Mine is easily accessible and has been thoroughly surveyed and described, so it is a good place to see what Lidar data can (and cannot) reveal. Choose a place you know!

All of the following is achieved by double-clicking on the DTM layer in the “Layers” panel then choosing the “Symbology” option. When you open this up, it will not look exactly as below, because the default “Render type” is the rather boring “Singleband gray”. Change this to “Hillshade”. Also change “Zoomed in” to “Cubic”; if you don’t do this, the appearance when zoomed in looks a little bit like linen (this is an easy thing to forget). You must either “Apply” the changes or “OK” to close the properties window for the settings to take effect.

To begin with, “Multidirectional” will be un-ticked. In that state you can play around with the altitude and azimuth (compass direction) of the imaginary sun for which the shading is created. It will quickly become clear that different features are revealed for different angles, but that if you are interested in showing several features, there isn’t a good answer. This is where multidirectional hillshade comes in; it combines the effect of several simulated suns (3 for QGIS). Since the hillshade algorithm uses gradient and aspect to determine the shade of grey, it will work if you have several 5km tiles loaded.

To combine the hillshade with either the OpenStreetMap, change the “Blending mode” from “Normal” to “Multiply”. This merges the DTM and the map images so that the map appears to be draped over a 3D surface. I find this the best way to interpret features in relation to landscape features. A similar effect can be obtained with the Bing aerial photography layer, although if there are buildings and trees in the area of interest, it may be better to use a DSM instead. Combining the DTM and aerial photographs can often be very revealing since the photographs can show variations in vegetation or surface material in topographically indifferent ground, whereas the DTM reveals aspects which are indifferent in ground cover.

An alternative to hillshading is to set the “Render type” to “Singleband pseudocolor” (the TIFF images are “single band” because there is a single value for each pixel, which corresponds with the height, whereas colour photographs usually have three bands with separate red, green, and blue values per pixel). The magic button is “Classify“! There are lots of colour schemes to experiment with – change “Color ramp” – but most are horrible.

The Min/Max values can be useful to limit the colour range to the elevation range in a particular area of interest. A 10m range can nicely bring out spoil heaps and hollows. These values can also be used to force several 5km tiles to have the same colour scale, so to appear seamless.

Most of the other options will not be useful for general experimentation, but the brightness/contrast/saturation slides can be useful for preparing images for print.

Some tricks:

  • Right-click on the DTM layer and choose “Duplicate Layer”. Now set one of the two layers to be hillshade and one to be pseudocolour. This can really bring out relief.
  • Make your own multidirectional hillshade by duplicating the DTM layer 2 or more times and setting the azimuth separately on each layer. This can help to reveal features which are not showing nicely with the QGIS fixed azimuth angles.

Here is a composite image of Magpie Mine using the DTM mentioned above (so the buildings have been magically transported away). The spoil heap near the 1869 engine house and the reservoir are easily seen, as are the main veins outside the heavily re-modelled central area, but can you make out the four gin circles and the crushing circle? Gin/crushing circles are much easier to see on the ground than from a DTM when the terrain is quite flat. How about the covered flue from the square chimney to the long engine house, the slime ponds and dressing area, and the straight tramroad from Dirty Redsoil into the centre of the main site? In this case, adding the pseudocolour actually makes it harder to make out the more subtle features where the changes in slope and aspect are more significant than changes in elevation.

Composite of Magpie Mine comprising OpenStreetMap base map overlain by a multidirectional hillshade and a single band pseudocolor constrained to the altitude range of 315m to 325m. (Click to open larger image)

QGIS can also generate contour lines from DTMs; change the “Render type” again. The contour options are fairly self-explanatory and work well down to as low as 1m interval with the National Lidar Programme data. The “index contour” can be used to make every 5th, 10th, etc contour be styled differently. The “input downscaling” setting default of 4 usually works well; this smooths out the contour lines to make them less jittery as the limits of the spatial resolution (and to a lesser degree the vertical accuracy) come into play. Larger values make for smoother contours.

I find there isn’t a single magic setting which works for all cases; different sites and different exploratory questions indicate different settings, but there is enough variety and power in what QGIS provides that there is usually a satisfactory approach, within the limits of the data.

Creating Terrain Profiles

Since the DEM encodes height on a grid, it is possible to construct profiles. The standard QGIS plugin “Profile tool” is the easiest way of doing this and there are a few more advanced plugins available. The tool is quite self-explanatory, so here is an example without a “how to”. The area around Wass’ Level (also known as Moorhigh New Level) on Maury Rake is in an accessible grassland location just South of the old Midland Railway bed through Millers Dale. A visit to the site allows the spoil heaps and settling ponds to be easily seen along with the presumed collapsed level at around 245m elevation, at the edge of the woodland. The Lidar image reveals some interesting features up-slope which cannot be seen from the open area but the elevation helpfully reveals enough to motivate a bit of struggling through the undergrowth. Could the feature at 255m be evidence for the excavation from the surface of a winze down from Wass’ Level to the now-lost Upper Level (destroyed when the railway was build just above it)? The paper entitled “The Maury and Burfoot Mines, Taddington and Brushfield, Derbyshire” by John Barnatt & Chris Heathcote in Mining History 15-3 describes the site and its history.

Aside: the vertical line near the right is just an artefact of having used two DEM tiles which have been separately hill-shaded; the hillshade algorithm can’t work out what the slope/aspect is at the edge of a tile. This can be avoided by combining the tiles into a single layer in QGIS, which should be done for publication but I’ve left it in here as there is educational value in showing and explaining the issue.

The Area Around Wass’ Level (SK149730) on Maury Rake, Millers Dale.
(Click for larger image)

Some Oddments…

Once you have got to grips with the basics, there are numerous plugins and processing algorithms to experiment with. The Visibility Analysis plugin uses the DTM to determine those parts of a map which show what can and cannot be seen from a vantage point. There are algorithms which will calculate surface slope and aspect, or roughness (Processing Toolbox > Raster Terrain Analysis); these values can then me mapped to colours using the same technique as above. Mapping slope to pseudocolour does a better job of revealing gin circles than hillshade when the terrain is quite level.

Using a WMS (Web Map Service) is generally more convenient than adding 5km square Tiff tiles in QGIS but the available services provide a combined hillshaded and pseudocoloured image and only a DTM is currently available (things might change). The separate services for 2m and 1m DTM (etc) are:

  • https://environment.data.gov.uk/spatialdata/lidar-composite-digital-terrain-model-dtm-1m-2020/wms
  • https://environment.data.gov.uk/spatialdata/lidar-composite-digital-terrain-model-dtm-2m-2020/wms

The following URL may be added as a WMS to discover the extent of coverage without the DEM: https://environment.data.gov.uk/spatialdata/survey-index-files/wms . NB this will not work in a web browser, but the same source can be accessed using an online service via web browser.

The elevation data is available (as opposed to a hillshaded and pseudocoloured image), but only using the ArcGIS Image Server protocol. In QGIS this requires the use of the “ArcGIS ImageServer Connector” plugin, which works but the image redraws rather slowly. The URL to use is of the form: https://environment.data.gov.uk/image/rest/services/SURVEY/LIDAR_Composite_2m_DTM_2020_Elevation/ImageServer.

We can hope that, once the National Lidar Programme has concluded, the WMS provision will improve, along with the rather confusing array of data download, WMS, and online map viewers. Quite a few things changed during the drafting of this article.

Credits

OpenStreetMap base maps have been produced from the OSM raster map tile service. This is © OpenStreetMap contributors and used under the terms of the Open Database Licence: https://opendatacommons.org/licenses/odbl/.

Digital terrain models used for creating profiles and DEM visualisations are from Defra, used under the Open Government Licence v3.0: http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/

Review of CTC Tools – Don’t Buy from CTC Tools!

The prices look good but CTC tools is hopeless.

I ordered some tools and got a tracking code, with a shipping date of Nov 25th. The tracking said the local courier had not yet received the package. After several weeks I queried this and was told that the package had in fact been returned due to an inadequate address. They send a pdf of the package label. The address was correct. The courier (Hermes) must just have been lazy. OK this may not be CTC’s fault but their failure to inform me about the return IS.

They did give me a refund but had taken payment from paypal in USD, in spite of the website being priced in £ sterling, so the refund came out £6.66 short due to the exchange rate changing. They refused to make up the difference, citing a policy (who reads those) which said prices were given in sterling “as a service”.

So, six weeks after shipping, I have no tools and am £6.66 the poorer for it.

No doubt they will protest that none of this is their fault, but it is just cause for poor reputation; I could have spend a few pounds extra and received my tools in good time. I suggest you do the same!

Searching Ordnance Survey Names for Evidence of Mining History

This article describes some informal/experimental work in which various sources of data were processed to find traces of mining history in the names of places.

A few of the results from the place name search, displayed over an Open Street Map

The Geographic Area

My area of interest is broadly-speaking the Peak District and adjacent areas. For practical purposes, I have defined both a detailed boundary and a rectangular bounding box. The latter takes in all of the former, which is defined using Parish (and similar town/urban) boundaries. Since the boundary of Sheffield stretches into outlying areas which are of interest, this way of defining the boundary ends up including the city and Eastern areas. The same is not the case for Greater Manchester. Noteworthy cases where areas outside the Peak District are included are: the Churnet Valley and Alderley Edge.

The bounding box is (E-min, N-min, E-max, N-max): (381355, 340577, 445079, 402069)

Coordinates will either use fully-numerical eastings and northings to 1m, or use 100km letters (SK covers most of the area) and a variable number of digits.

Where source data is chunked according to 100km or 20km tiles, the small excursion of the boundary north of 400000 is neglected.

Data Sources and Pre-processing

All data sources are ultimately from the Ordnance Survey, but with some qualifications:

  • OS Open Map Local* for NGR squares SJ and SK, was used, limited to shapefile data files “NamedPlace” and “Road”.
  • OS Open Names* is provided in 20km tiles; CSV files with the following names were used: SJ 84, 86, and 88; SK 04, 06, 08, 24, 26, 28.
  • OS 1:50,000 Scale Gazetteer is no longer available from OS but I had a copy from 2009 on disk. This contains names which are not present in currently-available OS downloads. It was processed to extract entries occurring in the same area as used for Open Names (the data includes the tile designator).
  • The Visions of Britain “GB 1900 Gazetteer” (the abridged version) was initially limited to the area extent then an attempt was made to remove entries which are descriptive of features (e.g. “Old Lead Mines”) rather than being names. This is a somewhat subjective exercise. Proper names for lead mines are quite common in this gazetteer; these were left in, although the main intent of the activity is to find place names which had arisen from mining activity, rather than finding named mines.

* – obtained from the OS Open Data download area.

A final spatial filter was applied in all cases to limit the input data to the “detailed area”.

Search Terms

The range of possible name-parts of interest is split up, largely according to variation-spellings of the same root, but with some “misc” categories which contain various thematically-related words. The categories/terms are given below, with the category name in bold.

  • coal: words starting with “coal”, “cole”, or “collier”. Historically, charcoal was also referred to this way.
  • pit: names ending, or having a word ending, in “pit” or “pits”.
  • lead: words beginning “lead”, “led”, “lyd”, “lidgat” or “lidyat”.
  • mine: words starting “mine”.
  • cost: words starting “coars”, “cost”, or “coast”. These could be due to Old English names indicating a mining trial (see PDMHS Bulletin 7-6 pp 339-341).
  • mining misc: a word beginning with one of “rake” or “raike”, “delf” or “delph”, “gin” or “engine”, or “ochre”.
  • bole: a word containing either “bole” or “brenn”. The latter catches “brenner”, the operator of a bole.
  • smelting misc: terms other than “bole”, which signify smelting: “sm.lt” (where . is any letter), “cupola”, “pig”, or “slag”.
  • misc: catches names with words containing “jagger”, “belland”, “forge”, or beginning with “copper”, “bloomery”, “furnace”, or “furnes”.

While it is clear that these search terms will include obviously-erroneous names, these are not excluded from the results; given the status of this work as “for interest and as stimulus”, this feels appropriate.

Search results were further processed to try to remove unwanted multiplicity arising from either the same name appearing in several sources, and from linear features such as roads having multiple entries. This was done by checking for identical names within 500m. This sometimes fails, as different sources may differ in capitalisation or make composite words such as “Bolehill”.

Results

Three map exports are available:

Three maps were created due to difficulties combining all the options using the Leaflet technology which underpins the web maps. For maps which do not immediately show the names, simply place the mouse cursor over a point.

Some Observations

Some of the search terms will give less-obvious false “hits”, and so all should be taken with some caution; place name specialists rely on written records from far earlier than those used here. Some example false friends are: “coal” might originate from “cold”, although we might reason that local geology makes the former more likely; “lydgate” is often though to derive from Old English “hlid-geat”, a swing gate. I will consider the case of “lyd” in a later post.

Almost all of the hits for “mine” are named mines, with a few exceptions such as “Miner’s Cottage”, “Miners Hill”, or “Miners’ Standard PH”.

Licences and Acknowledgements

This work includes data from:

The maps were created in QGIS using the QGIS2Web plugin and the Leaflet web mapping open source toolkit (see links at the bottom of the map screens).

The maps use data from Open Street Map and the National Library of Scotland, under an open licence for non-commercial use.

This blog post and the derived data created by me are licensed under Creative Commons BY-SA.

 

Adding British Geological Survey (BGS) WMS Layers to geojson.io

This took a little while to do; there doesn’t seem to be anywhere explaining how to add WMS layers and controls in simple terms… so here it is for anyone else. Pieced together from various places, the leaflet documentation, and some guesswork, here is how to add WMS layers for BGS solid and superficial deposit geology and linear features to a geojson.io view. The layer opacity has been set to 0.5 to allow the base map to be seen.

You will need to access the developers’ console (F12 on Firefox).

It will look like this:

Enter the following commands where the “>>” is:

var solid = L.tileLayer.wms("https://map.bgs.ac.uk/arcgis/services/BGS_Detailed_Geology/MapServer/WMSServer", {layers: 'BGS.50k.Bedrock', format: 'image/png', version: '1.3.0'});
var drift = L.tileLayer.wms("https://map.bgs.ac.uk/arcgis/services/BGS_Detailed_Geology/MapServer/WMSServer", {layers: 'BGS.50k.Superficial.deposits', format: 'image/png', version: '1.3.0'});
var linear = L.tileLayer.wms("https://map.bgs.ac.uk/arcgis/services/BGS_Detailed_Geology/MapServer/WMSServer", {layers: 'BGS.50k.Linear.features', format: 'image/png', version: '1.3.0'});
var layer_control = {"Solid": solid, "Drift": drift, "Linear Features": linear};
solid.setOpacity(0.5);
drift.setOpacity(0.5);
linear.setOpacity(0.5);
window.api.map.addLayer(drift);
window.api.map.addLayer(solid);
window.api.map.addLayer(linear);
L.control.layers(null, layer_control).addTo(window.api.map);

I find I have to leave the console now.

One thing to note is that the BGS WMS will not return an image if you are very “zoomed out”. Zoom in until the scale bar shows 500m or 1km.

 

Other BGS WMS layers to add:

var artificial = L.tileLayer.wms("https://map.bgs.ac.uk/arcgis/services/BGS_Detailed_Geology/MapServer/WMSServer", {layers: 'BGS.50k.Artificial.ground', format: 'image/png', version: '1.3.0'});
var movement = L.tileLayer.wms("https://map.bgs.ac.uk/arcgis/services/BGS_Detailed_Geology/MapServer/WMSServer", {layers: 'BGS.50k.Mass.movement', format: 'image/png', version: '1.3.0'});

Spreadsheet to Assist Melodeon Tuning

Here is a simple spreadsheet to assist with tuning two voiced melodeons: Melodeon Tuning Spreadsheet (Excel).

It has been set up for, and contains working data from, my D/G Hohner Pokerwork retune. I created it for two main purposes: 1) to plot the existing tuning; 2) to calculate the amount of de-tune for a Viennese tuning with an equal beat all the way up the scale (also known as Dedic tuning from Ian Dedic). I also used it to measure the tuning on a fairly new Serenellini melodeon with drier tuning, to get a better idea of what the tuning of a newish mid-range melodeon is like.

Some notes on using the spreadsheet for those who don’t want to find out by fiddling:

  • leave the “Notes Lookup” sheet alone; it contains the look-up from the piano key numbers to the note name and frequency.
  • On the sheets “G Row” and “D Row”:
    • change the entries in the “Piano Key” column if your button layout or keys are different; “Note Name” and “Concert Freq” change automatically.
    • If you want to find the tuning to achieve a constant beat frequency, alter the values in cells F3 and G3. I have chosen 4Hz (beats per second), which is in the tremolo range, and set this as -2Hz/+2Hz for Viennese tuning.
    • Alternatively… it is common for accordions to be tuning with tapering amounts of reed de-tuning going up the scale, with the beat increasing somewhat. Entering values into row K will compute the beat frequency in row L.
  • The sheets “… Measurements” should automatically populate the left-most columns. This should be mostly self-explanatory. I made columns to record measurements with the reed in the box and on the reed block on a tuning table, computing the difference (“delta”). Since tuning on the table is far easier than in the box, I use this to estimate a correction to my target tuning when using the table.
    • The “block hole number” is just a reference to my numbering of the holes on the chord block… its very confusing working out which reed is which!

 

I you are interested in melodeon tunings, or in the practicalities of tuning a melodeon, the melnet forums are essential reading.

Chinley Churn Underground Quarrying (Mines)

From a visit with Beth Knight earlier this year, here are the locations of some underground workings on Chinley Churn. This post is not a recommendation to visit; the workings contain dangerous loose rock and areas liable to collapse.

Point Name WGS84 Longitude WGS84 Latitude OS Grid Ref. 10m Altitude
1 -1.94595 53.34832 SK 0369 8349 407.5 m
2 -1.94561 53.34882 SK 0371 8355 416.6 m
3 -1.94563 53.34927 SK 0371 8360 415.2 m
4 -1.94546 53.34964 SK 0372 8364 423.6 m
5 -1.94625 53.35133 SK 0367 8382 422.1 m

Photographs showing the entrances, in the same order as the points above:

Chinley Churn Entrance 1

 

Chinley Churn Entrance 2

 

Chinley Churn Entrance 3

 

Chinley Churn Entrance 4

 

Chinley Churn Entrance 5

Decorative and Irregular PCBs in Eagle (with Inkscape)

Eagle does not lend itself to unusual, possibly artistic, board outlines copper tracks, silk screen etc. There are a number of Eagle ULPs (user language programs) that can import DXF files, and yet more which import bit maps as a mass of polygons, but I struggled to get any of them to work, or simply balked at the number of fiddly steps required. Basically: it was just too hard or error-prone. Then I stumbled across an article by Shabaz Yousaf complaining about the same problem but pointing to HPGL (Hewlett Packard Graphics Language) as a solution. He provides a C program to do the conversion but: a) I wanted to make a ULP for ease of use and b) I wanted to add some configurability at run-time. This article outlines the approach I took. I am assuming that images are created from the outset in Inkscape but Inkscape can import many formats, so it should be possible to use it on existing images, and any graphics software supporting HPGL output should work (HPGL appears to be very simple, but it may be the case that other software creates HPGL in a way my code does not recognise… so your mileage may vary!).

The User Language Program – hpgl2scr

The latest version is in my GitHub ULP repository (direct link to ULP). Please report bugs via GitHub if possible. Also any friendly guidance; this is my first ULP. This is available under an open source licence.

hpgl2scr

It should be fairly self-explanatory to use after a bit of messing about. Note that the wire (or polygon boundary) width auto-changes depending on the layer that is chosen. Adding more layers is easy by editing the ULP.

Some notes on usage:

  • HPGL files can contain several separate lines (think a pen moving about in an old-school plotter). Hpgl2scr creates one or more paths of wires or polygon boundaries in the specified layer.
  • A .scr file is created in the same folder as the .hpgl input. This file may be used to repeat the import.
  • In normal use choose the Wire output type. When using polygon output, note that setting the pour type = cutout may be used after import for subtractive effects (but only for copper layers).
  • All units are mm.
  • The imported image can be positioned in four ways:
  1. Aligned to the x and y axes with all wires at positive coordinate values, i.e. in the correct position for a board outline.
  2. Centred at a chosen position.
  3. Off-board (on the opposite side of the origin compared to option 1). This is the best place to group-select the shape and manually move it into place.
  4. Absolute positioning; (0,0) of the original graphic maps to (0,0) on the board. Absolute positioning may be useful if several layers are used in an image and each converted to separate HPGL files for separate importing. Beware that the origin for Inkscape is top left, whereas for boards in Eagle it is bottom left.
  • The Scale to Fit Box setting will rescale the image (preserving aspect ratio) so that it fits within a box of the specified size (in mm). This is quite useful in that it allows Inkscape to be used without worrying about absolute size, but the final board can be within the limits for PCB cost etc.

Using Copper Layers

As imported, copper layers do not form part of the circuit as far as Eagle is concerned. You cannot just connect to them and use them as pre-routed tracks (OK, so you can leave partially-routed tracks but dangling airwires just feels so untidy, and I always check for unrouted airwires so this would just be another cause of missed mistakes). The solution to this problem is to  set the name of the imported wire to be the same name as an existing net. Use the “Name” tool to find the name of the airwire you are interested in, and the Name tool again to change the name of the imported wire to match it. Once done, an airwire should appear from your imported wire. The you can now manually route a trace to the imported wire and it connects nicely.

Note: if you now do a “rip-up all”, your imported wire gets ripped up!

Hints and Tips for Inkscape

I recommend setting document properties to give mm as the default units (File > Document Properties).

inkscape_mmNote that the inkscape image elements MUST be saved as paths (vectors) to export to HPGL. Use the Path menu: Path > Object to Path. If your image does not appear in inkscape check this first!

Simply “save as”, selecting the HP Graphics Language file type.

inkscape_hpgl
Pen number and flatness are irrelevant. Other parameters should be as shown. Make sure resolution is 1016 dpi if the absolute size matters.

The thickness of the lines in inkscape is irrelevant; the HPGL just follows the path.

Being Ambitious

If you want to be ambitious and to compose several different elements (board outline, various bits of copper and silk) in a single inkscape image you will end up using layers in inkscape. Use a separate inkscape layer whenever the HPGL import parameters will differ (e.g. Eagle layer, wire vs polygon, wire thickness). Make all inkscape layers but one be invisible and export to HPGL, ensuring that the “Plot invisible layers” checkbox is not checked (see the “HPGL Output” screenshot above).

The import will probably be best done with the “absolute” positioning. It would be possible to import “off-board” and to move the elements into place, but positioning will not be easy to replicate in eagle. The trickiness arises because Inkscape uses the top left as origin (0,0), whereas Eagle uses the bottom left. This is easily fixed by checking the “Mirror Y-axis” option on export. Make sure that the image is right down in the bottom left corner so that ends up in roughly the right place in Eagle.

An Example

Here is a quickly executed (and slightly wonky) LED star for a “magic wand”. This followed the approach mentioned above of using 2 layers in inkscape. The inner and outer stars were positioned and sized using typed-in values in inkscape and imported one layer at a time with absolute positioning. The imported copper wire was re-named to GND (I had already named the net in the schematic).

wand

It look about 20 minutes, including some time forgetting most of the hints I’ve given!