SU_PYROW

kicad5-libraries-repos.sh

02-26-2018 By: SU_PYROW

kicad5-libraries-repos.sh is a script that was modified from the library-repos-install.sh from the KiCad source to help automate
downloading and updating of the KiCad-5.0-dev AKA KiCad-nightly Libraries including 3D, Schematic symbols, PCB footprints, Templates, update the symbol
and footprint tables. In addition it can also conveniently add your own custom library that you may have already created at the very end of those tables.

Library data are provided as compressed archives of the individual libraries within the following categories: These repositories are currently a work in progress.

Using this script to clone the library repositories means that you only need to download updates to the libraries, rather than re-downloading library files multiple times.
This can significantly save on bandwidth usage and allows you to control which library versions you use.

You must fill in the variables yourself!

You must also make the file executable before running with root/sudo capabilities:

$> chmod +x kicad5-libraries-repos.sh
$> sudo ./kicad5-libraries-repos.sh

This is the very script is use myself

#!/bin/bash
#
# Modified version of library-repos-install.sh from the KiCad source
# Customized for KiCad-5.0-dev AKA KiCad-nightly
# 02-26-2018  SU_PYROW
#
# 
# Git KiCad library repos:
#
# The "install_prerequisites" step is the only "distro dependent" one.  Could modify
# that step for other linux distros.
# This script requires "git".  The package bzr-git is not up to the task.
# The first time you run with option --install-or-update that is the slowest, because
# git clone from github.com is slow.
# After that updates should run faster.

# There are two reasons why you might want to run this script:
#
# 1) You want to contribute to the KiCad library team maintained libraries and have yet to
#    discover or have chosen not to use the COW feature in the Github "Plugin Type".
#
# 2) You want to run with local pretty footprint libraries and not those remotely located
#    on https://github.com using Github plugin. '
#
#	a) Change as needed for YOUR environment!

####    ################----------   ~/.bashrc   ----------################
####    export KISYSMOD="/electronics/KiCad/LIBRARIES/kicad-footprints"
####    export KICAD_SYMBOL_DIR="/electronics/KiCad/LIBRARIES/kicad-symbols"
####    export KISYS3DMOD="/electronics/KiCad/LIBRARIES/kicad-packages3D"
####    export KICAD_PTEMPLATES="/electronics/KiCad/LIBRARIES/kicad-templates"

####	OPTIONAL #export KISYSMINE="/electronics/KiCad/MY_LIBS"
####    ################----------     KiCad     ----------################


#	b} Do not forget to set Environmental Variables in 'Preferences->Configure Paths'

#	c) if you have your own library add environment variable KISYSMINE by UNCOMMENTING
#	   and be sure to scroll down to section: ' --install-or-update ' to add the library




#############     U S E  A T  Y O U R   O W N  R I S K      ###################

############################     VARIABLES    #################################
# Set where the library repos will go, use a full path

# NOTE:  FINAL WORKING DIRECTORY WILL GENERATE A FOLDER NAMED "LIBRARIES"
#  I.E.  /electronics/KiCad/LIBRARIES/  IF YOU USE THE FOLLOWING WORKING_TREES

WORKING_TREES=${WORKING_TREES:-/electronics/KiCad}
USERNAME=${USERNAME:-YOURUSERNAMEHERE}

usage()
{
    echo ""
    echo " usage:"
    echo ""
    echo "./kicad5-libraries-repos.sh "
    echo "    where  is one of:"
    echo "      --install-prerequisites     (install command tools needed here, run once first.)"
    echo "      --install-or-update         (from github, into $WORKING_TREES/LIBRARIES/. )"
    echo "      --remove-all-libraries      (remove all libraries from $WORKING_TREES/LIBRARIES/. )"
    echo "      --remove-orphaned-libraries (remove local footprint libraries which have been deleted or renamed at github.)"
    echo "      --list-footprint-libraries  (show the full list of github footprint libraries.)"
    echo "      --create-bat-file           (create kicad5-libraries-repos.bat file for Windows)"
    echo ""
    echo "	examples (with --install-prerequisites once first):"
    echo "	$ ./kicad5-libraries-repos.sh --install-prerequisites"
    echo "	$ ./kicad5-libraries-repos.sh --install-or-update"
}


install_prerequisites()
{
    # Find a package manager, PM
    PM=$( command -v yum || command -v apt-get )

    # assume all these Debian, Mint, Ubuntu systems have same prerequisites
    if [ "$(expr match "$PM" '.*\(apt-get\)')" == "apt-get" ]; then
        #echo "debian compatible system"
        sudo apt-get install git curl cut #sed


    # assume all yum systems have same prerequisites
    elif [ "$(expr match "$PM" '.*\(yum\)')" == "yum" ]; then
        #echo "red hat compatible system"
        # Note: if you find this list not to be accurate, please submit a patch:
        sudo yum install git curl cut #sed

            
    else
        echo
        echo "Incompatible System. Neither 'yum' nor 'apt-get' found. Not possible to"
        echo "continue. Please make sure to install git, curl, and cut before using this"
        echo "script."
        echo
        exit 1
    fi
}


detect_pretty_repos()
{
    # Check for the correct option to enable extended regular expressions in
    # sed. This is '-r' for GNU sed and '-E' for (older) BSD-like sed, as on
    # Mac OSX.
#    if [ $(echo | sed -r '' &>/dev/null; echo $?) -eq 0 ]; then
#        SED_EREGEXP="-r"
#    elif [ $(echo | sed -E '' &>/dev/null; echo $?) -eq 0 ]; then
#        SED_EREGEXP="-E"
#    else
#        echo "Your sed command does not support extended regular expressions. Cannot continue."
#        exit 1
#    fi

    # Use github API to list repos for org KiCad, then subset the JSON reply for only
    # *.pretty repos in the "full_name" variable."/KiCad/kicad-footprints/tree/master/Valve.pretty"
 #   repo_footprints=`curl -s "https://github.com/KiCad/kicad-footprints" 2> /dev/null \
 #       | sed $SED_EREGEXP 's:.+ "full_name".*"/KiCad/kicad-footprints/tree/master/(.+\.pretty)",:\1:p;d'`

    #echo "FP_REPOS:$FP_REPOS"
    
    FP_REPOS=`curl -s 'https://github.com/KiCad/kicad-footprints' 2> /dev/null \
    | grep -oP '(?<=href=")[^"]+.pretty(?=")' | cut -d'/' -f6-`

	#FP_REPOS=`echo $FP_REPOS | tr " " "\n" | sort`

   #echo "repo_footprints sorted:$repo_footprints"
}



update_symbols_libraries()
{
    if [ ! -d "$WORKING_TREES" ]; then
        sudo mkdir -p "$WORKING_TREES/LIBRARIES"
        echo " mark $WORKING_TREES/LIBRARIES as owned by me"
        sudo chown -R `whoami` "$WORKING_TREES/LIBRARIES"
    fi
    cd $WORKING_TREES

    if [ ! -e "$WORKING_TREES/LIBRARIES" ]; then
        mkdir -p "$WORKING_TREES/LIBRARIES"
    fi

    for repo_symbols in kicad-symbols; do
         #echo "repo_symbols:$repo_symbols"

        if [ ! -e "$WORKING_TREES/LIBRARIES/$repo_symbols" ]; then

            echo "installing $WORKING_TREES/LIBRARIES/$repo_symbols"

            git clone "https://github.com/KiCad/$repo_symbols" "$WORKING_TREES/LIBRARIES/$repo_symbols"
        else
            echo "updating $WORKING_TREES/LIBRARIES/$repo_symbols"
            cd "$WORKING_TREES/LIBRARIES/$repo_symbols"

            git pull
        fi
    done
}


update_3d_libraries()
{

      for repo_3d in kicad-packages3D; do
         #echo "repo_3d:$repo_3d"

        if [ ! -e "$WORKING_TREES/LIBRARIES/$repo_3d" ]; then

           echo "installing $WORKING_TREES/LIBRARIES/$repo_3d"

          git clone "https://github.com/KiCad/$repo_3d" "$WORKING_TREES/LIBRARIES/$repo_3d"
       else
           echo "updating $WORKING_TREES/LIBRARIES/$repo_3d"
           cd "$WORKING_TREES/LIBRARIES/$repo_3d"

           git pull
       fi
  done
}





update_footprint_libraries()
{
    for repo_footprints in kicad-footprints $FP_REPOS; do
         #echo "repo_footprints:$repo_footprints"

        if [ ! -e "$WORKING_TREES/LIBRARIES/$repo_footprints" ]; then

            echo "installing $WORKING_TREES/LIBRARIES/$repo_footprints"

            git clone "https://github.com/KiCad/$repo_footprints" "$WORKING_TREES/LIBRARIES/$repo_footprints"
        else
            echo "updating $WORKING_TREES/LIBRARIES/$repo_footprints"
            cd "$WORKING_TREES/LIBRARIES/$repo_footprints"

            git pull
        fi
    done
}


update_template_libraries()
{
    for repo_templates in kicad-templates; do
         #echo "repo_templates:$repo_templates"

        if [ ! -e "$WORKING_TREES/LIBRARIES/$repo_templates" ]; then

            echo "installing $WORKING_TREES/LIBRARIES/$repo_templates"

            git clone "https://github.com/KiCad/$repo_templates" "$WORKING_TREES/LIBRARIES/$repo_templates"
        else
            echo "updating $WORKING_TREES/LIBRARIES/$repo_templates"
            cd "$WORKING_TREES/LIBRARIES/$repo_templates"

            git pull
        fi
    done
}



listcontains()
{
    local list=$1
    local item=$2
    local ret=1
    local OIFS=$IFS

    # omit the space character from internal field separator.
    IFS=$'\n'

    for word in $list; do
        if [ "$word" == "$item" ]; then
            ret=0
            break
        fi
    done

    IFS=$OIFS
    return $ret
}


remove_orphaned_libraries()
{
    cd $WORKING_TREES/LIBRARIES/kicad-footprints

    if [ $? -ne 0 ]; then
        echo "Directory $WORKING_TREES/LIBRARIES/kicad-footprints does not exist."
        echo "The option --remove-orphaned-libraries should be used only after you've run"
        echo "the --install-or-update at least once."
        exit 2
    fi

    detect_pretty_repos

    for mylib in *.pretty; do
        echo "checking local lib: $mylib"

        if ! listcontains "$FP_REPOS" "$mylib"; then
            echo "Removing orphaned local library $WORKING_TREES/LIBRARIES/$mylib"
            rm -rf "$mylib"
        fi
    done
}


if [ $# -eq 1 -a "$1" == "--install-or-update" ]; then
    update_symbols_libraries
    update_footprint_libraries
    update_3d_libraries
    update_template_libraries
    echo "Installing KiCad Symbol and Footprint Library Tables = fp-lib-table and sym-lib-table to /home/$USERNAME/.config/kicad/"
    
    ##### THE COMMENTED OUT LINES ARE TO ADD A PERSONAL LIBRARY to fp-lib-table #####
    #####  COMMENT OUT  'cp ... '  and REMOVE on commented out 'head' and 'echo' lines to add a personal library.
    
    cp $WORKING_TREES/LIBRARIES/kicad-footprints/fp-lib-table /home/$USERNAME/.config/kicad/fp-lib-table;
    
    ##### THE COMMENTED OUT LINES ARE TO ADD A PERSONAL LIBRARY to fp-lib-table #####
        
   # head -n -1 $WORKING_TREES/LIBRARIES/kicad-footprints/fp-lib-table > /home/$USERNAME/.config/kicad/fp-lib-table;
   # echo '  (lib (name MyCustomParts)(type KiCad)(uri "$(KISYSMINE)/MyCustomParts.pretty")(options "")(descr "My Custom Component Footprints")))' >> /home/$USERNAME/.config/kicad/fp-lib-table;
    
    
    ##### THE COMMENTED OUT LINES ARE TO ADD A PERSONAL LIBRARY to sym-lib-table #####
    #####  COMMENT OUT  'cp ... '  and REMOVE on commented out 'head' and 'echo' lines to add a personal library.
    
    cp $WORKING_TREES/LIBRARIES/kicad-symbols/sym-lib-table /home/$USERNAME/.config/kicad/sym-lib-table;

   # head -n -1 $WORKING_TREES/LIBRARIES/kicad-symbols/sym-lib-table > /home/$USERNAME/.config/kicad/sym-lib-table
   #  echo '  (lib (name MyCustomComponents)(type Legacy)(uri ${KISYSMINE}/MyCustomLibrary/MyCustomComponents.lib)(options "")(descr "My Custom Schematic Symbols")))' >> /home/$USERNAME/.config/kicad/sym-lib-table;
    
    
    echo "Done installing/updating KiCad Symbol and Footprint Library Tables"
    exit
fi


if [ $# -eq 1 -a "$1" == "--remove-orphaned-libraries" ]; then
    remove_orphaned_libraries
    exit
fi


if [ $# -eq 1 -a "$1" == "--remove-all-libraries" ]; then
    rm -rf "$WORKING_TREES/LIBRARIES"
    exit
fi


if [ $# -eq 1 -a "$1" == "--install-prerequisites" ]; then
    install_prerequisites
    exit
fi

if [ $# -eq 1 -a "$1" == "--list-footprint-libraries" ]; then

    # use github API to get repos into PRETTY_REPOS var
    detect_pretty_repos

    #for repo_footprints in $FP_REPOS; do
    echo "$FP_REPOS"
    #done

    exit
fi

# may re-direct this output to a disk file for Windows *.BAT file creation.
if [ $# -eq 1 -a "$1" == "--create-bat-file" ]; then


        echo "REM This file was created using kicad5-libraries-repos.sh on linux."
	echo "REM Run it from a directory you desire as the base for all libraries."
	echo ""
	echo "REM Do not forget to set Environmental Variables in 'Preferences->Configure Paths'"
	echo "REM and 'Advanced System Settings' in the Control Panel"		
	echo ""	
	echo "REM KISYSMOD to folder where kicad-footprints folder is located"
	echo "REM KICAD_SYMBOL_DIR to folder where kicad-symbols folder is located"
	echo "REM KISYS3DMOD to folder where kicad-packages3D folder is located"
	echo "REM KICAD_PTEMPLATES to folder where kicad-templates folder is located"
	echo ""
	echo "git clone https://github.com/KiCad/kicad-footprints"
	echo "git clone https://github.com/KiCad/kicad-symbols"
	echo "git clone https://github.com/KiCad/kicad-packages3D"
	echo "git clone https://github.com/KiCad/kicad-templates"
	echo ""
	echo "REM if Environmental Variables are set and files have already been downloaded"
	echo "REM COMMENT the 'git' commands above and UNCOMMENT the git commands below."
	echo "REM Run from the directory you previously desired as the base for all libraries."
	echo ""
	echo "#git pull https://github.com/KiCad/kicad-footprints"
	echo "#git pull https://github.com/KiCad/kicad-symbols"
	echo "#git pull https://github.com/KiCad/kicad-packages3D"
	echo "#git pull https://github.com/KiCad/kicad-templates"
	
	
        echo "REM This file was created using kicad5-libraries-repos.sh on linux." >kicad5-libraries-repos.bat;
	echo "REM Run it from a directory you desire as the base for all libraries." >>kicad5-libraries-repos.bat;
	echo "" >>kicad5-libraries-repos.bat;
	echo "REM Do not forget to set Environmental Variables in 'Preferences->Configure Paths'" >>kicad5-libraries-repos.bat;
	echo "REM and 'Advanced System Settings' in the Control Panel" >>kicad5-libraries-repos.bat;		
	echo "" >>kicad5-libraries-repos.bat;	
	echo "REM KISYSMOD to folder where kicad-footprints folder is located" >>kicad5-libraries-repos.bat;
	echo "REM KICAD_SYMBOL_DIR to folder where kicad-symbols folder is located" >>kicad5-libraries-repos.bat;
	echo "REM KISYS3DMOD to folder where kicad-packages3D folder is located" >>kicad5-libraries-repos.bat;
	echo "REM KICAD_PTEMPLATES to folder where kicad-templates folder is located" >>kicad5-libraries-repos.bat;
	echo "" >>kicad5-libraries-repos.bat;
	echo "git clone https://github.com/KiCad/kicad-footprints" >>kicad5-libraries-repos.bat;
	echo "git clone https://github.com/KiCad/kicad-symbols" >>kicad5-libraries-repos.bat;
	echo "git clone https://github.com/KiCad/kicad-packages3D" >>kicad5-libraries-repos.bat;
	echo "git clone https://github.com/KiCad/kicad-templates" >>kicad5-libraries-repos.bat;
	echo "" >>kicad5-libraries-repos.bat;
	echo "REM if Environmental Variables are set and files have already been downloaded" >>kicad5-libraries-repos.bat;
	echo "REM COMMENT the 'git' commands above and UNCOMMENT the git commands below." >>kicad5-libraries-repos.bat;
	echo "REM Run from the directory you previously desired as the base for all libraries." >>kicad5-libraries-repos.bat;
	echo "" >>kicad5-libraries-repos.bat;
	echo "#git pull https://github.com/KiCad/kicad-footprints" >>kicad5-libraries-repos.bat;
	echo "#git pull https://github.com/KiCad/kicad-symbols" >>kicad5-libraries-repos.bat;
	echo "#git pull https://github.com/KiCad/kicad-packages3D" >>kicad5-libraries-repos.bat;
	echo "#git pull https://github.com/KiCad/kicad-templates" >>kicad5-libraries-repos.bat;
         
        
        exit
fi

usage

This document was last modified on: 133120 13:08:26 UTC