[index] [home]

Tweet

KiCad with generated ".net" and ".cmp" files



This text describes how to generate KiCad netlists and component-assignments files programmatically using a domain-specific language (DSL) on top of Tcl.

Background

In an earlier page, ways in which KiCad eeschema netlist- (.net) and cvpcb component-files (.cmp) can be simplified were given.

The idea then was to eventually be able to generate both these files programmatically, thus skipping the schematic capture and component-assignment stages.

This can be useful when a project contains repetitive or hierarchical circuitry. Although KiCad itself has a concept of "hierarchical circuit", it can be cumbersome having to spend space on schematic-pages for very similar circuitry, and having to provide glue-wiring between these blocks.

Furthermore, I don't think changes in hierarchical circuits propagate up to already existing instances - I don't think hierarchical circuits were meant to be used like macros, but more as a way to abstract a single subcircuit at a time.

Instead, circuits and subcircuits can be modeled using something resembling procedures and macros in programming languages - which is what this page is about.

Caveat

Merged schematic capture and component-assignment

In the KiCad workflow, eeschema is the schematic capture tool, and cvpcb is the component-assignment tool.

Schematic symbols are in principle separate from corresponding physical packages. Associating footprints with schematics symbols works by means of matching pin-numbers.

In generating everything up to the layout-phase programmatically, I chose to merge schematic symbol and assigned footprint in something called a "physical component".

The idea here is that abstraction can be used instead, to accomplish a better separation between footprints and building-blocks used for modeling. (See the NPN-transistor abstraction later on for an example.)

Learn by example: an inverter out of discrete components

A logic inverter can be constructed out of resistors and a transistor as follows:

In the normal KiCad workflow, the resistor and transistor could be drawn in a hierarchical circuit, which could then be instantiated multiple times.

Using the tool described here, it can be described programmatically instead:

    virtual component "inverter" with pins { A Y Vcc GND } consists of {

        resistor_1k "Rc" { 
            { pin 1 at Vcc } 
            { pin 2 at Y   } }

        resistor_100 "Rs" { { pin 1 at A } }

        transistor_NPN "Q" { 
            { pin b at Rs:2 }
            { pin c at Y    }
            { pin e at GND  } }
    }

This defines a new part called "inverter" to be used in a circuit-description. (I chose for a verbose sort of language, with risk of making it look like LOLCODE.)

Explanation:

So, the "inverter" clause creates a new super-component as shown in the picture.

Physical and virtual components

The inverter defined above is called a virtual component - simply said, it does not occur in the BOM, because it consists of other components.

Although virtual components can consist of other virtual components (e.g. an oscillator containing among other parts an inverter), at the lowest level, only so-called physical components make up the circuit.

In a circuit-description, a physical component has no statements describing its inner workings, simply because it's already "finished" as-is. A physical component has a value (currently not used, but meant for clarity and BOM-generation later) and a footprint (e.g. "SM0603" in KiCad-lingo).

As an example, in the description of the inverter above, a physical component "resistor_100" is instantiated as base-resistor (Rs):

    ...                    
    resistor_100 "Rs" { { pin 1 at A } }
    ...                    

The definition of "resistor_100" is as follows:

    physical component "resistor_100" with pins { 1 2 } has value 100 and footprint "SM0603"

This should speak for itself: "resistor_100" is a 100 Ohm 0603 resistor, to be instantiated elsewhere.

Encapsulation to hide footprint-details

The inverter-example uses a NPN-transistor (Q) as switching element.

In KiCad

This is where KiCad starts to leak: during schematic capture, the user doesn't want to be concerned with the physical choice of transistor, and likely wants to use a generic "NPN transistor" symbol for all NPN-transistors. This symbol has its pins numbered from 1 to 3.

However, different physical transistors have different packages, and even within a package (e.g. SOT-23), transistor-terminals may not always be exposed at the same package-lead.

Since there is no additional layer mapping schematic symbols to footprints, this means that either multiple schematic symbols or multiple footprints have to be used, with only difference between them being the pin-assignment. (Having multiple footprints probably makes more sense than having multiple schematic symbols here.)

DISCLAIMER: I may be misunderstanding the KiCad-way, but fact is that there exist e.g. different "SOT-23" footprints to accomodate for differences in pin-numbering. I can't see any way of making this work without an additional mapping-layer, which KiCad to my knowledge does not have.

In this tool...

...the choice was made to create that missing abstraction-layer. That is, physical components can be wrapped inside another component-description with the only effect being a renumbering or renaming of the pins.

As an example, the given inverter was described using a generic "transistor_NPN" component:

    ...
    transistor_NPN "Q" { 
        { pin b at Rs:2 }
        { pin c at Y    }
        { pin e at GND  } }
    ...

This subcomponent has pins named b, c and e with obvious meanings, and is described as follows:

    virtual component "transistor_NPN" with pins { b c e } consists of {

        bc847 "Q" { 
            { pin 1 at b }
            { pin 2 at e }
            { pin 3 at c } }
    }

Thus, underlying each instance of "transistor_NPN" is a physical component (BC847):

    physical component "bc847" with pins { 1 2 3 } has value "bc847" and footprint "SOT23"

The pin-numbering of this physical component follows the actual pin-numbering as given in its data-sheet, as can be seen here:

So virtual component "transistor_NPN" hides the actual pin-numbering of the physical component. In case another physical transistor is used instead, only the definition of "transistor_NPN" has to be changed accordingly.

(This method has shortcomings too, but works reasonably well for simple but large schematics.)

Complete description of "inverter"

The complete description of inverter, followed by a single instantiation (as U1) and instructions to write netlist and component-assignments file looks like this:

    physical component "resistor_100" with pins { 1 2 } has value 100 and footprint "SM0603"

    physical component "resistor_1k" with pins { 1 2 } has value "1k" and footprint "SM0603"

    physical component "bc847" with pins { 1 2 3 } has value "bc847" and footprint "SOT23"



    virtual component "transistor_NPN" with pins { b c e } consists of {

        bc847 "Q" { 
            { pin 1 at b }
            { pin 2 at e }
            { pin 3 at c } }
    }



    virtual component "inverter" with pins { A Y Vcc GND } consists of {

        resistor_1k "Rc" { 
            { pin 1 at Vcc } 
            { pin 2 at Y   } }

        resistor_100 "Rs" { { pin 1 at A } }

        transistor_NPN "Q" { 
            { pin b at Rs:2 }
            { pin c at Y    }
            { pin e at GND  } }
    }



    physical component "testpad" with pin 1 has value "test" and footprint "TESTPAD"



    # ##########################################################################################



    testpad "P1" { { pin 1 at power } }
    testpad "P2" { { pin 1 at input } }
    testpad "P3" { { pin 1 at output } }
    testpad "P4" { { pin 1 at ground } }

    inverter "U1" { 
        { pin A   at input  }
        { pin Y   at output }
        { pin Vcc at power  }
        { pin GND at ground } }



    write_kicad_netlist /tmp/kicad_test/kicad_test.net
    write_kicad_cmplist /tmp/kicad_test/kicad_test.cmp

What's great about this, is that this is valid Tcl code! In other words, parsing of the DSL is done implicitly.

Generated netlist and component-assignments file

The resulting netlist (*.net) generated by the tool looks as follows:

    (export (version D)
    (components
    (comp (ref P1) (value test))
    (comp (ref P2) (value test))
    (comp (ref P3) (value test))
    (comp (ref P4) (value test))
    (comp (ref U1_Rc) (value 1k))
    (comp (ref U1_Rs) (value 100))
    (comp (ref U1_Q_Q) (value bc847))
    )
    (nets
    (net (code 1) (name "power")
    (node (ref P1) (pin 1))
    (node (ref U1_Rc) (pin 1))
    )
    (net (code 2) (name "input")
    (node (ref P2) (pin 1))
    (node (ref U1_Rs) (pin 1))
    )
    (net (code 3) (name "output")
    (node (ref P3) (pin 1))
    (node (ref U1_Rc) (pin 2))
    (node (ref U1_Q_Q) (pin 3))
    )
    (net (code 4) (name "ground")
    (node (ref P4) (pin 1))
    (node (ref U1_Q_Q) (pin 2))
    )
    (net (code 5) (name "")
    (node (ref U1_Rs) (pin 2))
    (node (ref U1_Q_Q) (pin 1))
    )
    )
    )

As can be seen, top-level names from the circuit-description such as "output" and "ground" occur as net-names in the netlist. This helps during the layout-stage. (Internal nets are not named, but each net is always given a unique code.)

The refdes-naming (e.g. "U1_Rs") gives a hint as to the location of the component in the circuit-hierarchy. This also helps while layouting, making it easier to group nearby components together.

Resulting component-assignments file is as follows:

    Cmp-Mod V01

    BeginCmp
    Reference = P1;
    IdModule  = TESTPAD;
    EndCmp

    BeginCmp
    Reference = P2;
    IdModule  = TESTPAD;
    EndCmp

    BeginCmp
    Reference = P3;
    IdModule  = TESTPAD;
    EndCmp

    BeginCmp
    Reference = P4;
    IdModule  = TESTPAD;
    EndCmp

    BeginCmp
    Reference = U1_Rc;
    IdModule  = SM0603;
    EndCmp

    BeginCmp
    Reference = U1_Rs;
    IdModule  = SM0603;
    EndCmp

    BeginCmp
    Reference = U1_Q_Q;
    IdModule  = SOT23;
    EndCmp

    EndListe

LOLworthy anecdote: apparently my KiCad-version (2013-may-18 stable) cares about whitespace.

This worked...

    IdModule  = SM0603;

...while this did not:

    IdModule = SM0603;

(spot the difference). I am guessing this is fixed in more recent versions.

Resulting ratsnest

The ratsnest when reading these files into pcbnew looks like this after moving components around a bit:

Generator source-code

Not too pretty (lots of globals and very imperative), but it works:

    #!/usr/bin/env tclsh



    variable nets {}



    proc make_proc { name arglist body args } {

        set body [ string map $args $body ]

        proc $name $arglist $body
    }



    proc val_or_default { _var default } { 

        expr { [ uplevel #0 info exists $_var ] ? [ uplevel #0 set $_var ] : $default } 
    }



    proc lpop _li {

        upvar 1 $_li li

        set li [ lreplace $li end end ]
    }



    proc refdes_path_get {} { val_or_default refdes_path {} }



    proc refdes_path_push refdes { uplevel #0 lappend refdes_path $refdes }



    proc refdes_path_pop {} { uplevel #0 lpop refdes_path }



    proc component_add_virtual refdes {

        global components

        dict set components $refdes {}
    }



    proc component_add_physical { refdes value footprint } { 

        global components

        dict set components $refdes value     $value
        dict set components $refdes footprint $footprint
    }



    proc set_component_footprint { compname footprint } { uplevel #0 dict set component_footprints $compname $footprint }



    proc add_connection_to_netlist { new_pin existing_pin } {

        global nets

        for { set i 0 } { $i < [ llength $nets ] } { incr i } {

            if { $existing_pin in [ lindex $nets $i ] } {

                # Net for existing pin found; add new pin to that net.

                lset nets $i end+1 $new_pin

                return
            }
        }

        # No net for existing pin exists yet; create net, and add both existing and new pins to it.

        lappend nets [ list $new_pin $existing_pin ]
    }



    proc make_connections { connections refdes } {

        set parent_refdes_path [ refdes_path_get ]
        refdes_path_push $refdes

        foreach connection $connections {

            # Local pin always exists in the component being instantiated.

            set local_pin [ concat [ refdes_path_get ] [ lindex $connection end-2 ] ]

            # Existing pin can exist in previously defined sibling or in parent.

            set pin_string [ lindex $connection end ]

            if { [ regexp {^(\w+):(\w+)$} $pin_string -> sibling sib_pin ] } {

                set existing_pin [ concat $parent_refdes_path $sibling $sib_pin ]

            } else {

                set existing_pin [ concat $parent_refdes_path $pin_string ]
            }

            # Add local pin to proper net (and optionally create net for existing pin on-the-fly).

            add_connection_to_netlist $local_pin $existing_pin
        }

        refdes_path_pop
    }



    proc virtual { 'component' compname 'with' 'pins' pins 'consists' 'of' code } {

        make_proc $compname { refdes { connections {} } } {

            component_add_virtual [ concat [ refdes_path_get ] $refdes ]

            make_connections $connections $refdes

            refdes_path_push $refdes

            %CODE%

            refdes_path_pop
        } \
            %CODE%     $code
    }



    proc physical { 'component' compname 'with' 'pins' pins 'has' 'value' value 'and' 'footprint' footprint } {

        set_component_footprint $compname $footprint

        make_proc $compname { refdes { connections {} } } {

            component_add_physical [ concat [ refdes_path_get ] $refdes ] %VALUE% %FOOTPRINT%

            make_connections $connections $refdes
        } \
            %VALUE%     $value \
            %FOOTPRINT% $footprint
    }



    proc get_physical_component_properties refdes {

        global components

        if { ! ( $refdes in $components ) } { error "refdes '$refdes' not found in component-list (bug)" }

        dict get $components $refdes
    }



    proc stringified_refdes refdes { join $refdes _ }



    proc physical_components {} {

        global components

        set refdeses {}

        dict for { refdes comp_prop } $components {

            if { [ llength $comp_prop ] } {

                lappend refdeses $refdes
            }
        }

        return $refdeses
    }



    # $kicad_net_info = { $name, $nodes = { $refdes_string, $pin_nr }* }*

    proc kicad_net_info net {

        set net_info [ dict create name "" nodes {} ]

        foreach node $net {

            if { [ llength $node ] == 1 } {

                # Net-entry is a single word, so use it as the net-name.

                dict set net_info name $node

            } else {

                # Multi-word net-entry - could either be a physical or virtual component & pin.

                set refdes [ lrange $node 0 end-1 ]

                set comp_prop [ get_physical_component_properties $refdes ]

                if { [ llength $comp_prop ] } {

                    # This refdes & pin belong to a physical component, so include it in our info.

                    dict lappend net_info nodes [ dict create        \
                        refdes_string [ stringified_refdes $refdes ] \
                        pin_nr        [ lindex $node end ]           \
                    ]
                }
            }
        }

        return $net_info
    }



    proc write_kicad_cmplist fname {

        set fd [ open $fname w ]

        puts $fd "Cmp-Mod V01"
        puts $fd ""

        foreach refdes [ physical_components ] {

            set comp_prop [ get_physical_component_properties $refdes ]

            puts $fd "BeginCmp"
            puts $fd "Reference = [ stringified_refdes $refdes ];"
            puts $fd "IdModule  = [ dict get $comp_prop footprint ];"
            puts $fd "EndCmp"
            puts $fd ""
        }

        puts $fd "EndListe"

        close $fd
    }



    proc write_kicad_netlist fname {

        set fd [ open $fname w ]

        puts $fd "(export (version D)"



        puts $fd "(components"

        foreach r [ physical_components ] {

            puts -nonewline $fd "(comp "
            puts -nonewline $fd "(ref [ stringified_refdes $r ]) "
            puts $fd "(value [ dict get [ get_physical_component_properties $r ] value ]))"
        }

        puts $fd ")"



        puts $fd "(nets"

        set netcode 0

        upvar #0 nets nets

        foreach net $nets {

            set net_info [ kicad_net_info $net ]

            puts -nonewline $fd "(net "
            puts -nonewline $fd "(code [ incr netcode ]) "
            puts $fd "(name \"[ dict get $net_info name ]\")"

            foreach node [ dict get $net_info nodes ] {

                puts -nonewline $fd "(node "
                puts -nonewline $fd "(ref [ dict get $node refdes_string ]) "
                puts $fd "(pin [ dict get $node pin_nr        ]))"
            }

            puts $fd ")"
        }

        puts $fd ")"



        puts $fd ")"

        close $fd
    }

That's all!


Delivered to you by Vim, GNU Make, MultiMarkdown, bozohttpd, NetBSD, and 1 human.