読者です 読者をやめる 読者になる 読者になる

戦場のプログラマー

名前は誰も知らない。

Raspberry Pi(Raspbian) に jq コマンドをインストールしてみた

こんな特殊な人はいないと思うけど、一応メモ。 プロセッサのアーキテクチャが異なるため、Linux 用のバイナリを落としてきても以下の様なメッセージが表示されて動作しない。

pi@raspberrypi ~ $ jq
-bash: /usr/local/bin/jq: cannot execute binary file

恐れる必要はありません、ポータブル C で書かれているので jq のソースをクローンしてきてビルドすればOKです。

TL;DR

ものすごい辛い道のりでした。Raspbian に jq 入れるべきではありません。 ビルドした Raspbian 用のバイナリはあるので、 jq 使いたいだけの人はこれを入れたほうがいいかもです。

http://wnoguchi-public.s3.amazonaws.com/bin/raspbian/jq

長い道のりの始まり

git clone https://github.com/stedolan/jq.git

まずは AutoTools のインストールが必要となる。

sudo apt-get -y install autoconf

しかし

autoreconf -i

としても以下のようなエラーで怒られてしまう。

pi@raspberrypi ~/jq $ autoreconf -i
configure.ac:14: installing `config/ar-lib'
configure.ac:22: installing `config/compile'
configure.ac:12: installing `config/install-sh'
configure.ac:12: installing `config/missing'
Makefile.am:35: Libtool library used but `LIBTOOL' is undefined
Makefile.am:35:   The usual way to define `LIBTOOL' is to add `LT_INIT'
Makefile.am:35:   to `configure.ac' and run `aclocal' and `autoconf' again.
Makefile.am:35:   If `LT_INIT' is in `configure.ac', make sure
Makefile.am:35:   its definition is in aclocal's search path.
Makefile.am: installing `config/depcomp'
configure.ac: installing `config/ylwrap'
autoreconf: automake failed with exit status: 1

Libtool を入れる。

sudo apt-get -y install libtool

再度トライ。

pi@raspberrypi ~/jq $ autoreconf -i
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `config'.
libtoolize: copying file `config/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `config/m4'.
libtoolize: copying file `config/m4/libtool.m4'
libtoolize: copying file `config/m4/ltoptions.m4'
libtoolize: copying file `config/m4/ltsugar.m4'
libtoolize: copying file `config/m4/ltversion.m4'
libtoolize: copying file `config/m4/lt~obsolete.m4'
configure.ac:14: installing `config/ar-lib'
configure.ac:22: installing `config/compile'
configure.ac:21: installing `config/config.guess'
configure.ac:21: installing `config/config.sub'
configure.ac:12: installing `config/install-sh'
configure.ac:12: installing `config/missing'
Makefile.am: installing `config/depcomp'
configure.ac: installing `config/ylwrap'

OKみたい。お次は configure 。

pi@raspberrypi ~/jq $ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking for gcc option to accept ISO C99... -std=gnu99
checking for gcc -std=gnu99 option to accept ISO Standard C... (cached) -std=gnu99
checking how to run the C preprocessor... gcc -std=gnu99 -E
checking for bison... no
checking for byacc... no
checking build system type... armv6l-unknown-linux-gnueabihf
checking host system type... armv6l-unknown-linux-gnueabihf
checking how to print strings... printf
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc -std=gnu99... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert armv6l-unknown-linux-gnueabihf file names to armv6l-unknown-linux-gnueabihf format... func_convert_
op
checking how to convert armv6l-unknown-linux-gnueabihf file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc -std=gnu99 object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc -std=gnu99 supports -fno-rtti -fno-exceptions... no
checking for gcc -std=gnu99 option to produce PIC... -fPIC -DPIC
checking if gcc -std=gnu99 PIC flag -fPIC -DPIC works... yes
checking if gcc -std=gnu99 static flag -static works... yes
checking if gcc -std=gnu99 supports -c -o file.o... yes
checking if gcc -std=gnu99 supports -c -o file.o... (cached) yes
checking whether the gcc -std=gnu99 linker (/usr/bin/ld) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether gcc -std=gnu99 and cc understand -c and -o together... yes
checking bison version... not bison
checking for flex... no
checking for lex... no
checking oniguruma.h usability... no
checking oniguruma.h presence... no
checking for oniguruma.h... no
checking for valgrind... no
configure: WARNING: valgrind is required to test jq.
checking for memmem... yes
checking for bundle... no
checking for Ruby dependencies... configure: WARNING:
*****************************************************************
*  Ruby dependencies for building jq documentation not found.   *
*  You can still build, install and hack on jq, but the manpage *
*  will not be rebuilt and some of the tests won't run.         *
*  See docs/README.md for how to install the docs dependencies. *
*****************************************************************
no
checking for isatty... yes
checking for _isatty... no
checking for thread-local storage... yes
checking whether byte ordering is bigendian... no
configure: creating ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands

man ページが作られないよみたいな警告が出ているが気にしない。

pi@raspberrypi ~/jq $ make -j8
  LEX    lexer.c
/bin/bash: flex: command not found
  YACC   parser.c
Makefile:1275: recipe for target 'lexer.c' failed
make: *** [lexer.c] Error 127
make: *** Waiting for unfinished jobs....
./config/ylwrap: line 113: yacc: command not found
Makefile:664: recipe for target 'parser.c' failed
make: *** [parser.c] Error 1

いろんなものが足りてない。Debian 系はあまり慣れてないからかいろいろ躓く。

sudo apt-get -y install bison

改めて

pi@raspberrypi ~/jq $ make -j8
  YACC   parser.c
  LEX    lexer.c
/bin/bash: flex: command not found
Makefile:1275: recipe for target 'lexer.c' failed
make: *** [lexer.c] Error 127
make: *** Waiting for unfinished jobs....
/home/pi/jq/parser.y:100.1-11: invalid directive: `%precedence'
/home/pi/jq/parser.y:101.1-11: invalid directive: `%precedence'
/home/pi/jq/parser.y:102.1-11: invalid directive: `%precedence'
/home/pi/jq/parser.y:274.1-6: invalid directive: `%empty'
/home/pi/jq/parser.y:288.1-6: invalid directive: `%empty'
/home/pi/jq/parser.y:296.1-6: invalid directive: `%empty'
/home/pi/jq/parser.y:535.1-6: invalid directive: `%empty'
/home/pi/jq/parser.y:702.1-6: invalid directive: `%empty'
Makefile:664: recipe for target 'parser.c' failed
make: *** [parser.c] Error 1

またエラー。

jq no longer builds on Ubuntu 12.04 · Issue #488 · stedolan/jq

を見ると俺は bison 3.x を使ってビルドしてるんだけど、どうよ? 的なコメントが書いてあり、
bison をソースからビルドして解決したみたいなことが書いてある。
バージョンを確認してみる。

pi@raspberrypi ~/jq $ bison --version
bison (GNU Bison) 2.5
Written by Robert Corbett and Richard Stallman.

Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

本当にありがとうございました。 だんだん jq を Raspbian で無理して使う必要性を感じなくなってくる。
正直 Raspberry Pi でこういうのをソースからコンパイルするのは負荷が大きい。めっちゃ遅い。

Bison のインストール

まずさっきパッケージで入れた Bison を消す。

sudo apt-get -y remove bison

そして以下から Bison のソースを落としてくる。

GNU Project Archives

wget http://ftp.gnu.org/gnu/bison/bison-3.0.1.tar.gz
tar xvzf bison-3.0.1.tar.gz

INSTALL を読む。そしてビルド、インストール。
このぐらい低レベルななインストーラになると依存性のエラーもあまりなくてよさげ。

./configure
make
make check

が、

pi@raspberrypi ~/bison-3.0.1 $ make check
  GEN      public-submodule-commit
make  check-recursive
make[1]: Entering directory '/home/pi/bison-3.0.1'
Making check in po
make[2]: Entering directory '/home/pi/bison-3.0.1/po'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/home/pi/bison-3.0.1/po'
Making check in runtime-po
make[2]: Entering directory '/home/pi/bison-3.0.1/runtime-po'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/home/pi/bison-3.0.1/runtime-po'
Making check in .
make[2]: Entering directory '/home/pi/bison-3.0.1'
  GEN      doc/bison.help
make  examples/calc++/calc++ examples/mfcalc/mfcalc examples/rpcalc/rpcalc ./tests/bison tests/atconfig tests/atlocal
make[3]: Entering directory '/home/pi/bison-3.0.1'
  YACC     examples/calc++/calc++-parser.stamp
  CXX      examples/calc++/examples_calc___calc__-calc++-driver.o
  LEX      examples/calc++/calc++-scanner.cc
  CXX      examples/calc++/examples_calc___calc__-calc++-scanner.o
g++: error: ./examples/calc++/calc++-scanner.cc: No such file or directory
g++: fatal error: no input files
compilation terminated.
Makefile:4023: recipe for target 'examples/calc++/examples_calc___calc__-calc++-scanner.o' failed
make[3]: *** [examples/calc++/examples_calc___calc__-calc++-scanner.o] Error 4
make[3]: Leaving directory '/home/pi/bison-3.0.1'
Makefile:4830: recipe for target 'check-am' failed
make[2]: *** [check-am] Error 2
make[2]: Leaving directory '/home/pi/bison-3.0.1'
Makefile:4379: recipe for target 'check-recursive' failed
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory '/home/pi/bison-3.0.1'
Makefile:4833: recipe for target 'check' failed
make: *** [check] Error 2

華麗にスルー。気にしてたら長生きできない。 たぶん Bison 使ってコアなことやろうとするとエラーはいて死ぬだろうけど・・・。

sudo -i
cd ~pi/bison-3.0.1
make install

そしてバージョン確認。

root@raspberrypi:/home/pi/bison-3.0.1# bison --version
bison (GNU Bison) 3.0.1
Written by Robert Corbett and Richard Stallman.

Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

OK。一般ユーザーに戻る。

jq コンパイル再開

pi@raspberrypi ~/jq $ make -j8
  LEX    lexer.c
/bin/bash: flex: command not found
Makefile:1275: recipe for target 'lexer.c' failed
make: *** [lexer.c] Error 127
make: *** Waiting for unfinished jobs....

flex 入れる。

sudo apt-get -y install flex
builtin.c:4:23: fatal error: oniguruma.h: No such file or directory
compilation terminated.
Makefile:654: recipe for target 'builtin.lo' failed
make[1]: *** [builtin.lo] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/pi/jq'
Makefile:488: recipe for target 'all' failed
make: *** [all] Error 2

鬼車の開発ヘッダ入れる。

sudo apt-get -y install libonig-dev

もう一回

./.libs/libjq.a(builtin.o): In function `f_match':
/home/pi/jq/builtin.c:613: undefined reference to `onig_new'
/home/pi/jq/builtin.c:630: undefined reference to `onig_region_new'
/home/pi/jq/builtin.c:632: undefined reference to `onig_search'
/home/pi/jq/builtin.c:707: undefined reference to `onig_foreach_name'
/home/pi/jq/builtin.c:711: undefined reference to `onig_region_free'
/home/pi/jq/builtin.c:725: undefined reference to `onig_region_free'
/home/pi/jq/builtin.c:729: undefined reference to `onig_free'
/home/pi/jq/builtin.c:608: undefined reference to `OnigEncodingUTF8'
/home/pi/jq/builtin.c:608: undefined reference to `OnigSyntaxPerl_NG'
/home/pi/jq/builtin.c:618: undefined reference to `onig_error_code_to_str'
/home/pi/jq/builtin.c:718: undefined reference to `onig_error_code_to_str'
collect2: ld returned 1 exit status
Makefile:610: recipe for target 'jq' failed
make[1]: *** [jq] Error 1
make[1]: Leaving directory '/home/pi/jq'
Makefile:488: recipe for target 'all' failed
make: *** [all] Error 2

configure やり直してみる。

./configure
make -j8

よさそう。

pi@raspberrypi ~/jq $ make check
make  check-am
make[1]: Entering directory '/home/pi/jq'
make  check-TESTS
make[2]: Entering directory '/home/pi/jq'
make[3]: Entering directory '/home/pi/jq'
PASS: tests/all.test
=============
1 test passed
=============
make[3]: Leaving directory '/home/pi/jq'
make[2]: Leaving directory '/home/pi/jq'
make[1]: Leaving directory '/home/pi/jq'

インストールする。

pi@raspberrypi ~/jq $ sudo make install
make  install-am
make[1]: Entering directory '/home/pi/jq'
make[2]: Entering directory '/home/pi/jq'
 /bin/mkdir -p '/usr/local/lib'
 /bin/bash ./libtool   --mode=install /usr/bin/install -c   libjq.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libjq.so.1.0.4 /usr/local/lib/libjq.so.1.0.4
libtool: install: (cd /usr/local/lib && { ln -s -f libjq.so.1.0.4 libjq.so.1 || { rm -f libjq.so.1 && ln -s libjq.so.1.0.4 libjq.so.1; }; })
libtool: install: (cd /usr/local/lib && { ln -s -f libjq.so.1.0.4 libjq.so || { rm -f libjq.so && ln -s libjq.so.1.0.4 libjq.so; }; })
libtool: install: /usr/bin/install -c .libs/libjq.lai /usr/local/lib/libjq.la
libtool: install: /usr/bin/install -c .libs/libjq.a /usr/local/lib/libjq.a
libtool: install: chmod 644 /usr/local/lib/libjq.a
libtool: install: ranlib /usr/local/lib/libjq.a
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /usr/local/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `LD_LIBRARY_PATH' environment variable
     during execution
   - add LIBDIR to the `LD_RUN_PATH' environment variable
     during linking
   - use the `-Wl,-rpath -Wl,LIBDIR' linker flag
   - have your system administrator add LIBDIR to `/etc/ld.so.conf'

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
 /bin/mkdir -p '/usr/local/bin'
  /bin/bash ./libtool   --mode=install /usr/bin/install -c jq '/usr/local/bin'
libtool: install: /usr/bin/install -c jq /usr/local/bin/jq
 /bin/mkdir -p '/usr/local/share/doc/jq'
 /usr/bin/install -c -m 644 README.md COPYING AUTHORS README '/usr/local/share/doc/jq'
 /bin/mkdir -p '/usr/local/include'
 /usr/bin/install -c -m 644 jv.h jq.h '/usr/local/include'
 /bin/mkdir -p '/usr/local/share/man/man1'
 /usr/bin/install -c -m 644 jq.1 '/usr/local/share/man/man1'
make[2]: Leaving directory '/home/pi/jq'
make[1]: Leaving directory '/home/pi/jq'

バージョン確認する。

pi@raspberrypi ~/jq $ which jq
/usr/local/bin/jq
pi@raspberrypi ~/jq $ jq --version
jq-1.4-148-g89791a0

使ってみる。

f:id:wnoguchi0727:20141109232617p:plain

イェーイ。

結論

Raspbian に jq を入れるのは根性が必要。

後世のために Raspberry Pi の Raspbian 用の jq バイナリを置いておきました。 好きに使ってください。

http://wnoguchi-public.s3.amazonaws.com/bin/raspbian/jq

References

  1. stedolan/jq
  2. Download jq
  3. AutoTools - Debian Wiki
  4. Autotoolsについてのメモ
  5. make: yacc: Command not found building libpcap - HowtoForge Forums | HowtoForge - Linux Howtos and Tutorials
  6. jq no longer builds on Ubuntu 12.04 · Issue #488 · stedolan/jq
  7. GNU Project Archives