blob: a05c12b7e700972fe5417dc6f933d419efcf6746 [file] [log] [blame]
Masahiro Yamada5a27c732015-05-20 11:36:07 +09001#!/usr/bin/env python2
2#
3# Author: Masahiro Yamada <yamada.masahiro@socionext.com>
4#
5# SPDX-License-Identifier: GPL-2.0+
6#
7
8"""
9Move config options from headers to defconfig files.
10
11Since Kconfig was introduced to U-Boot, we have worked on moving
12config options from headers to Kconfig (defconfig).
13
14This tool intends to help this tremendous work.
15
16
17Usage
18-----
19
20This tool takes one input file. (let's say 'recipe' file here.)
21The recipe describes the list of config options you want to move.
22Each line takes the form:
23<config_name> <type> <default>
24(the fields must be separated with whitespaces.)
25
26<config_name> is the name of config option.
27
28<type> is the type of the option. It must be one of bool, tristate,
29string, int, and hex.
30
31<default> is the default value of the option. It must be appropriate
32value corresponding to the option type. It must be either y or n for
33the bool type. Tristate options can also take m (although U-Boot has
34not supported the module feature).
35
36You can add two or more lines in the recipe file, so you can move
37multiple options at once.
38
39Let's say, for example, you want to move CONFIG_CMD_USB and
40CONFIG_SYS_TEXT_BASE.
41
42The type should be bool, hex, respectively. So, the recipe file
43should look like this:
44
45 $ cat recipe
46 CONFIG_CMD_USB bool n
47 CONFIG_SYS_TEXT_BASE hex 0x00000000
48
Joe Hershberger96464ba2015-05-19 13:21:17 -050049Next you must edit the Kconfig to add the menu entries for the configs
50you are moving.
51
Masahiro Yamada5a27c732015-05-20 11:36:07 +090052And then run this tool giving the file name of the recipe
53
54 $ tools/moveconfig.py recipe
55
56The tool walks through all the defconfig files to move the config
57options specified by the recipe file.
58
59The log is also displayed on the terminal.
60
61Each line is printed in the format
62<defconfig_name> : <action>
63
64<defconfig_name> is the name of the defconfig
65(without the suffix _defconfig).
66
67<action> shows what the tool did for that defconfig.
68It looks like one of the followings:
69
70 - Move 'CONFIG_... '
71 This config option was moved to the defconfig
72
73 - Default value 'CONFIG_...'. Do nothing.
74 The value of this option is the same as default.
75 We do not have to add it to the defconfig.
76
77 - 'CONFIG_...' already exists in Kconfig. Do nothing.
78 This config option is already defined in Kconfig.
79 We do not need/want to touch it.
80
81 - Undefined. Do nothing.
82 This config option was not found in the config header.
83 Nothing to do.
84
85 - Failed to process. Skip.
86 An error occurred during processing this defconfig. Skipped.
87 (If -e option is passed, the tool exits immediately on error.)
88
89Finally, you will be asked, Clean up headers? [y/n]:
90
91If you say 'y' here, the unnecessary config defines are removed
92from the config headers (include/configs/*.h).
93It just uses the regex method, so you should not rely on it.
94Just in case, please do 'git diff' to see what happened.
95
96
97How does it works?
98------------------
99
100This tool runs configuration and builds include/autoconf.mk for every
101defconfig. The config options defined in Kconfig appear in the .config
102file (unless they are hidden because of unmet dependency.)
103On the other hand, the config options defined by board headers are seen
104in include/autoconf.mk. The tool looks for the specified options in both
105of them to decide the appropriate action for the options. If the option
106is found in the .config or the value is the same as the specified default,
107the option does not need to be touched. If the option is found in
108include/autoconf.mk, but not in the .config, and the value is different
109from the default, the tools adds the option to the defconfig.
110
111For faster processing, this tool handles multi-threading. It creates
112separate build directories where the out-of-tree build is run. The
113temporary build directories are automatically created and deleted as
114needed. The number of threads are chosen based on the number of the CPU
115cores of your system although you can change it via -j (--jobs) option.
116
117
118Toolchains
119----------
120
121Appropriate toolchain are necessary to generate include/autoconf.mk
122for all the architectures supported by U-Boot. Most of them are available
123at the kernel.org site, some are not provided by kernel.org.
124
125The default per-arch CROSS_COMPILE used by this tool is specified by
126the list below, CROSS_COMPILE. You may wish to update the list to
127use your own. Instead of modifying the list directly, you can give
128them via environments.
129
130
131Available options
132-----------------
133
134 -c, --color
135 Surround each portion of the log with escape sequences to display it
136 in color on the terminal.
137
Joe Hershberger91040e82015-05-19 13:21:19 -0500138 -d, --defconfigs
139 Specify a file containing a list of defconfigs to move
140
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900141 -n, --dry-run
142 Peform a trial run that does not make any changes. It is useful to
143 see what is going to happen before one actually runs it.
144
145 -e, --exit-on-error
146 Exit immediately if Make exits with a non-zero status while processing
147 a defconfig file.
148
Joe Hershberger2144f882015-05-19 13:21:20 -0500149 -H, --headers-only
150 Only cleanup the headers; skip the defconfig processing
151
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900152 -j, --jobs
153 Specify the number of threads to run simultaneously. If not specified,
154 the number of threads is the same as the number of CPU cores.
155
156To see the complete list of supported options, run
157
158 $ tools/moveconfig.py -h
159
160"""
161
162import fnmatch
163import multiprocessing
164import optparse
165import os
166import re
167import shutil
168import subprocess
169import sys
170import tempfile
171import time
172
173SHOW_GNU_MAKE = 'scripts/show-gnu-make'
174SLEEP_TIME=0.03
175
176# Here is the list of cross-tools I use.
177# Most of them are available at kernel.org
178# (https://www.kernel.org/pub/tools/crosstool/files/bin/), except the followings:
179# arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases
180# blackfin: http://sourceforge.net/projects/adi-toolchain/files/
181# nds32: http://osdk.andestech.com/packages/
182# nios2: https://sourcery.mentor.com/GNUToolchain/subscription42545
183# sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu
184CROSS_COMPILE = {
185 'arc': 'arc-linux-',
186 'aarch64': 'aarch64-linux-',
187 'arm': 'arm-unknown-linux-gnueabi-',
188 'avr32': 'avr32-linux-',
189 'blackfin': 'bfin-elf-',
190 'm68k': 'm68k-linux-',
191 'microblaze': 'microblaze-linux-',
192 'mips': 'mips-linux-',
193 'nds32': 'nds32le-linux-',
194 'nios2': 'nios2-linux-gnu-',
195 'openrisc': 'or32-linux-',
196 'powerpc': 'powerpc-linux-',
197 'sh': 'sh-linux-gnu-',
198 'sparc': 'sparc-linux-',
199 'x86': 'i386-linux-'
200}
201
202STATE_IDLE = 0
203STATE_DEFCONFIG = 1
204STATE_AUTOCONF = 2
Joe Hershberger96464ba2015-05-19 13:21:17 -0500205STATE_SAVEDEFCONFIG = 3
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900206
207ACTION_MOVE = 0
208ACTION_DEFAULT_VALUE = 1
209ACTION_ALREADY_EXIST = 2
210ACTION_UNDEFINED = 3
211
212COLOR_BLACK = '0;30'
213COLOR_RED = '0;31'
214COLOR_GREEN = '0;32'
215COLOR_BROWN = '0;33'
216COLOR_BLUE = '0;34'
217COLOR_PURPLE = '0;35'
218COLOR_CYAN = '0;36'
219COLOR_LIGHT_GRAY = '0;37'
220COLOR_DARK_GRAY = '1;30'
221COLOR_LIGHT_RED = '1;31'
222COLOR_LIGHT_GREEN = '1;32'
223COLOR_YELLOW = '1;33'
224COLOR_LIGHT_BLUE = '1;34'
225COLOR_LIGHT_PURPLE = '1;35'
226COLOR_LIGHT_CYAN = '1;36'
227COLOR_WHITE = '1;37'
228
229### helper functions ###
230def get_devnull():
231 """Get the file object of '/dev/null' device."""
232 try:
233 devnull = subprocess.DEVNULL # py3k
234 except AttributeError:
235 devnull = open(os.devnull, 'wb')
236 return devnull
237
238def check_top_directory():
239 """Exit if we are not at the top of source directory."""
240 for f in ('README', 'Licenses'):
241 if not os.path.exists(f):
242 sys.exit('Please run at the top of source directory.')
243
244def get_make_cmd():
245 """Get the command name of GNU Make.
246
247 U-Boot needs GNU Make for building, but the command name is not
248 necessarily "make". (for example, "gmake" on FreeBSD).
249 Returns the most appropriate command name on your system.
250 """
251 process = subprocess.Popen([SHOW_GNU_MAKE], stdout=subprocess.PIPE)
252 ret = process.communicate()
253 if process.returncode:
254 sys.exit('GNU Make not found')
255 return ret[0].rstrip()
256
257def color_text(color_enabled, color, string):
258 """Return colored string."""
259 if color_enabled:
260 return '\033[' + color + 'm' + string + '\033[0m'
261 else:
262 return string
263
264def log_msg(color_enabled, color, defconfig, msg):
265 """Return the formated line for the log."""
266 return defconfig[:-len('_defconfig')].ljust(37) + ': ' + \
267 color_text(color_enabled, color, msg) + '\n'
268
269def update_cross_compile():
270 """Update per-arch CROSS_COMPILE via enviroment variables
271
272 The default CROSS_COMPILE values are available
273 in the CROSS_COMPILE list above.
274
275 You can override them via enviroment variables
276 CROSS_COMPILE_{ARCH}.
277
278 For example, if you want to override toolchain prefixes
279 for ARM and PowerPC, you can do as follows in your shell:
280
281 export CROSS_COMPILE_ARM=...
282 export CROSS_COMPILE_POWERPC=...
283 """
284 archs = []
285
286 for arch in os.listdir('arch'):
287 if os.path.exists(os.path.join('arch', arch, 'Makefile')):
288 archs.append(arch)
289
290 # arm64 is a special case
291 archs.append('aarch64')
292
293 for arch in archs:
294 env = 'CROSS_COMPILE_' + arch.upper()
295 cross_compile = os.environ.get(env)
296 if cross_compile:
297 CROSS_COMPILE[arch] = cross_compile
298
299def cleanup_one_header(header_path, patterns, dry_run):
300 """Clean regex-matched lines away from a file.
301
302 Arguments:
303 header_path: path to the cleaned file.
304 patterns: list of regex patterns. Any lines matching to these
305 patterns are deleted.
306 dry_run: make no changes, but still display log.
307 """
308 with open(header_path) as f:
309 lines = f.readlines()
310
311 matched = []
312 for i, line in enumerate(lines):
313 for pattern in patterns:
314 m = pattern.search(line)
315 if m:
316 print '%s: %s: %s' % (header_path, i + 1, line),
317 matched.append(i)
318 break
319
320 if dry_run or not matched:
321 return
322
323 with open(header_path, 'w') as f:
324 for i, line in enumerate(lines):
325 if not i in matched:
326 f.write(line)
327
328def cleanup_headers(config_attrs, dry_run):
329 """Delete config defines from board headers.
330
331 Arguments:
332 config_attrs: A list of dictionaris, each of them includes the name,
333 the type, and the default value of the target config.
334 dry_run: make no changes, but still display log.
335 """
336 while True:
337 choice = raw_input('Clean up headers? [y/n]: ').lower()
338 print choice
339 if choice == 'y' or choice == 'n':
340 break
341
342 if choice == 'n':
343 return
344
345 patterns = []
346 for config_attr in config_attrs:
347 config = config_attr['config']
348 patterns.append(re.compile(r'#\s*define\s+%s\W' % config))
349 patterns.append(re.compile(r'#\s*undef\s+%s\W' % config))
350
351 for (dirpath, dirnames, filenames) in os.walk('include'):
352 for filename in filenames:
353 if not fnmatch.fnmatch(filename, '*~'):
354 cleanup_one_header(os.path.join(dirpath, filename), patterns,
355 dry_run)
356
357### classes ###
358class KconfigParser:
359
360 """A parser of .config and include/autoconf.mk."""
361
362 re_arch = re.compile(r'CONFIG_SYS_ARCH="(.*)"')
363 re_cpu = re.compile(r'CONFIG_SYS_CPU="(.*)"')
364
365 def __init__(self, config_attrs, options, build_dir):
366 """Create a new parser.
367
368 Arguments:
369 config_attrs: A list of dictionaris, each of them includes the name,
370 the type, and the default value of the target config.
371 options: option flags.
372 build_dir: Build directory.
373 """
374 self.config_attrs = config_attrs
375 self.options = options
376 self.build_dir = build_dir
377
378 def get_cross_compile(self):
379 """Parse .config file and return CROSS_COMPILE.
380
381 Returns:
382 A string storing the compiler prefix for the architecture.
383 """
384 arch = ''
385 cpu = ''
386 dotconfig = os.path.join(self.build_dir, '.config')
387 for line in open(dotconfig):
388 m = self.re_arch.match(line)
389 if m:
390 arch = m.group(1)
391 continue
392 m = self.re_cpu.match(line)
393 if m:
394 cpu = m.group(1)
395
396 assert arch, 'Error: arch is not defined in %s' % defconfig
397
398 # fix-up for aarch64
399 if arch == 'arm' and cpu == 'armv8':
400 arch = 'aarch64'
401
402 return CROSS_COMPILE.get(arch, '')
403
Joe Hershberger96464ba2015-05-19 13:21:17 -0500404 def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900405 """Parse .config, defconfig, include/autoconf.mk for one config.
406
407 This function looks for the config options in the lines from
408 defconfig, .config, and include/autoconf.mk in order to decide
409 which action should be taken for this defconfig.
410
411 Arguments:
412 config_attr: A dictionary including the name, the type,
413 and the default value of the target config.
414 defconfig_lines: lines from the original defconfig file.
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900415 autoconf_lines: lines from the include/autoconf.mk file.
416
417 Returns:
418 A tupple of the action for this defconfig and the line
419 matched for the config.
420 """
421 config = config_attr['config']
422 not_set = '# %s is not set' % config
423
424 if config_attr['type'] in ('bool', 'tristate') and \
425 config_attr['default'] == 'n':
426 default = not_set
427 else:
428 default = config + '=' + config_attr['default']
429
Joe Hershberger96464ba2015-05-19 13:21:17 -0500430 for line in defconfig_lines:
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900431 line = line.rstrip()
432 if line.startswith(config + '=') or line == not_set:
433 return (ACTION_ALREADY_EXIST, line)
434
435 if config_attr['type'] in ('bool', 'tristate'):
436 value = not_set
437 else:
438 value = '(undefined)'
439
440 for line in autoconf_lines:
441 line = line.rstrip()
442 if line.startswith(config + '='):
443 value = line
444 break
445
446 if value == default:
447 action = ACTION_DEFAULT_VALUE
448 elif value == '(undefined)':
449 action = ACTION_UNDEFINED
450 else:
451 action = ACTION_MOVE
452
453 return (action, value)
454
455 def update_defconfig(self, defconfig):
456 """Parse files for the config options and update the defconfig.
457
458 This function parses the given defconfig, the generated .config
459 and include/autoconf.mk searching the target options.
460 Move the config option(s) to the defconfig or do nothing if unneeded.
461 Also, display the log to show what happened to this defconfig.
462
463 Arguments:
464 defconfig: defconfig name.
465 """
466
467 defconfig_path = os.path.join('configs', defconfig)
468 dotconfig_path = os.path.join(self.build_dir, '.config')
469 autoconf_path = os.path.join(self.build_dir, 'include', 'autoconf.mk')
470 results = []
471
472 with open(defconfig_path) as f:
473 defconfig_lines = f.readlines()
474
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900475 with open(autoconf_path) as f:
476 autoconf_lines = f.readlines()
477
478 for config_attr in self.config_attrs:
479 result = self.parse_one_config(config_attr, defconfig_lines,
Joe Hershberger96464ba2015-05-19 13:21:17 -0500480 autoconf_lines)
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900481 results.append(result)
482
483 log = ''
484
485 for (action, value) in results:
486 if action == ACTION_MOVE:
487 actlog = "Move '%s'" % value
488 log_color = COLOR_LIGHT_GREEN
489 elif action == ACTION_DEFAULT_VALUE:
490 actlog = "Default value '%s'. Do nothing." % value
491 log_color = COLOR_LIGHT_BLUE
492 elif action == ACTION_ALREADY_EXIST:
493 actlog = "'%s' already defined in Kconfig. Do nothing." % value
494 log_color = COLOR_LIGHT_PURPLE
495 elif action == ACTION_UNDEFINED:
496 actlog = "Undefined. Do nothing."
497 log_color = COLOR_DARK_GRAY
498 else:
499 sys.exit("Internal Error. This should not happen.")
500
501 log += log_msg(self.options.color, log_color, defconfig, actlog)
502
503 # Some threads are running in parallel.
504 # Print log in one shot to not mix up logs from different threads.
505 print log,
506
507 if not self.options.dry_run:
Joe Hershberger96464ba2015-05-19 13:21:17 -0500508 with open(dotconfig_path, 'a') as f:
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900509 for (action, value) in results:
510 if action == ACTION_MOVE:
511 f.write(value + '\n')
512
513 os.remove(os.path.join(self.build_dir, 'include', 'config', 'auto.conf'))
514 os.remove(autoconf_path)
515
516class Slot:
517
518 """A slot to store a subprocess.
519
520 Each instance of this class handles one subprocess.
521 This class is useful to control multiple threads
522 for faster processing.
523 """
524
525 def __init__(self, config_attrs, options, devnull, make_cmd):
526 """Create a new process slot.
527
528 Arguments:
529 config_attrs: A list of dictionaris, each of them includes the name,
530 the type, and the default value of the target config.
531 options: option flags.
532 devnull: A file object of '/dev/null'.
533 make_cmd: command name of GNU Make.
534 """
535 self.options = options
536 self.build_dir = tempfile.mkdtemp()
537 self.devnull = devnull
538 self.make_cmd = (make_cmd, 'O=' + self.build_dir)
539 self.parser = KconfigParser(config_attrs, options, self.build_dir)
540 self.state = STATE_IDLE
541 self.failed_boards = []
542
543 def __del__(self):
544 """Delete the working directory
545
546 This function makes sure the temporary directory is cleaned away
547 even if Python suddenly dies due to error. It should be done in here
548 because it is guranteed the destructor is always invoked when the
549 instance of the class gets unreferenced.
550
551 If the subprocess is still running, wait until it finishes.
552 """
553 if self.state != STATE_IDLE:
554 while self.ps.poll() == None:
555 pass
556 shutil.rmtree(self.build_dir)
557
558 def add(self, defconfig):
559 """Assign a new subprocess for defconfig and add it to the slot.
560
561 If the slot is vacant, create a new subprocess for processing the
562 given defconfig and add it to the slot. Just returns False if
563 the slot is occupied (i.e. the current subprocess is still running).
564
565 Arguments:
566 defconfig: defconfig name.
567
568 Returns:
569 Return True on success or False on failure
570 """
571 if self.state != STATE_IDLE:
572 return False
573 cmd = list(self.make_cmd)
574 cmd.append(defconfig)
575 self.ps = subprocess.Popen(cmd, stdout=self.devnull)
576 self.defconfig = defconfig
577 self.state = STATE_DEFCONFIG
578 return True
579
580 def poll(self):
581 """Check the status of the subprocess and handle it as needed.
582
583 Returns True if the slot is vacant (i.e. in idle state).
584 If the configuration is successfully finished, assign a new
585 subprocess to build include/autoconf.mk.
586 If include/autoconf.mk is generated, invoke the parser to
587 parse the .config and the include/autoconf.mk, and then set the
588 slot back to the idle state.
589
590 Returns:
591 Return True if the subprocess is terminated, False otherwise
592 """
593 if self.state == STATE_IDLE:
594 return True
595
596 if self.ps.poll() == None:
597 return False
598
599 if self.ps.poll() != 0:
600
601 print >> sys.stderr, log_msg(self.options.color,
602 COLOR_LIGHT_RED,
603 self.defconfig,
604 "failed to process.")
605 if self.options.exit_on_error:
606 sys.exit("Exit on error.")
607 else:
608 # If --exit-on-error flag is not set,
609 # skip this board and continue.
610 # Record the failed board.
611 self.failed_boards.append(self.defconfig)
612 self.state = STATE_IDLE
613 return True
614
615 if self.state == STATE_AUTOCONF:
616 self.parser.update_defconfig(self.defconfig)
Joe Hershberger96464ba2015-05-19 13:21:17 -0500617
618 """Save off the defconfig in a consistent way"""
619 cmd = list(self.make_cmd)
620 cmd.append('savedefconfig')
621 self.ps = subprocess.Popen(cmd, stdout=self.devnull,
622 stderr=self.devnull)
623 self.state = STATE_SAVEDEFCONFIG
624 return False
625
626 if self.state == STATE_SAVEDEFCONFIG:
627 defconfig_path = os.path.join(self.build_dir, 'defconfig')
628 shutil.move(defconfig_path,
629 os.path.join('configs', self.defconfig))
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900630 self.state = STATE_IDLE
631 return True
632
633 cross_compile = self.parser.get_cross_compile()
634 cmd = list(self.make_cmd)
635 if cross_compile:
636 cmd.append('CROSS_COMPILE=%s' % cross_compile)
Joe Hershberger7740f652015-05-19 13:21:18 -0500637 cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900638 cmd.append('include/config/auto.conf')
639 self.ps = subprocess.Popen(cmd, stdout=self.devnull)
640 self.state = STATE_AUTOCONF
641 return False
642
643 def get_failed_boards(self):
644 """Returns a list of failed boards (defconfigs) in this slot.
645 """
646 return self.failed_boards
647
648class Slots:
649
650 """Controller of the array of subprocess slots."""
651
652 def __init__(self, config_attrs, options):
653 """Create a new slots controller.
654
655 Arguments:
656 config_attrs: A list of dictionaris containing the name, the type,
657 and the default value of the target CONFIG.
658 options: option flags.
659 """
660 self.options = options
661 self.slots = []
662 devnull = get_devnull()
663 make_cmd = get_make_cmd()
664 for i in range(options.jobs):
665 self.slots.append(Slot(config_attrs, options, devnull, make_cmd))
666
667 def add(self, defconfig):
668 """Add a new subprocess if a vacant slot is found.
669
670 Arguments:
671 defconfig: defconfig name to be put into.
672
673 Returns:
674 Return True on success or False on failure
675 """
676 for slot in self.slots:
677 if slot.add(defconfig):
678 return True
679 return False
680
681 def available(self):
682 """Check if there is a vacant slot.
683
684 Returns:
685 Return True if at lease one vacant slot is found, False otherwise.
686 """
687 for slot in self.slots:
688 if slot.poll():
689 return True
690 return False
691
692 def empty(self):
693 """Check if all slots are vacant.
694
695 Returns:
696 Return True if all the slots are vacant, False otherwise.
697 """
698 ret = True
699 for slot in self.slots:
700 if not slot.poll():
701 ret = False
702 return ret
703
704 def show_failed_boards(self):
705 """Display all of the failed boards (defconfigs)."""
706 failed_boards = []
707
708 for slot in self.slots:
709 failed_boards += slot.get_failed_boards()
710
711 if len(failed_boards) > 0:
712 msg = [ "The following boards were not processed due to error:" ]
713 msg += failed_boards
714 for line in msg:
715 print >> sys.stderr, color_text(self.options.color,
716 COLOR_LIGHT_RED, line)
717
718def move_config(config_attrs, options):
719 """Move config options to defconfig files.
720
721 Arguments:
722 config_attrs: A list of dictionaris, each of them includes the name,
723 the type, and the default value of the target config.
724 options: option flags
725 """
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900726 if len(config_attrs) == 0:
727 print 'Nothing to do. exit.'
728 sys.exit(0)
729
730 print 'Move the following CONFIG options (jobs: %d)' % options.jobs
731 for config_attr in config_attrs:
732 print ' %s (type: %s, default: %s)' % (config_attr['config'],
733 config_attr['type'],
734 config_attr['default'])
735
Joe Hershberger91040e82015-05-19 13:21:19 -0500736 if options.defconfigs:
737 defconfigs = [line.strip() for line in open(options.defconfigs)]
738 for i, defconfig in enumerate(defconfigs):
739 if not defconfig.endswith('_defconfig'):
740 defconfigs[i] = defconfig + '_defconfig'
741 if not os.path.exists(os.path.join('configs', defconfigs[i])):
742 sys.exit('%s - defconfig does not exist. Stopping.' %
743 defconfigs[i])
744 else:
745 # All the defconfig files to be processed
746 defconfigs = []
747 for (dirpath, dirnames, filenames) in os.walk('configs'):
748 dirpath = dirpath[len('configs') + 1:]
749 for filename in fnmatch.filter(filenames, '*_defconfig'):
750 defconfigs.append(os.path.join(dirpath, filename))
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900751
752 slots = Slots(config_attrs, options)
753
754 # Main loop to process defconfig files:
755 # Add a new subprocess into a vacant slot.
756 # Sleep if there is no available slot.
757 for defconfig in defconfigs:
758 while not slots.add(defconfig):
759 while not slots.available():
760 # No available slot: sleep for a while
761 time.sleep(SLEEP_TIME)
762
763 # wait until all the subprocesses finish
764 while not slots.empty():
765 time.sleep(SLEEP_TIME)
766
767 slots.show_failed_boards()
768
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900769def bad_recipe(filename, linenum, msg):
770 """Print error message with the file name and the line number and exit."""
771 sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
772
773def parse_recipe(filename):
774 """Parse the recipe file and retrieve the config attributes.
775
776 This function parses the given recipe file and gets the name,
777 the type, and the default value of the target config options.
778
779 Arguments:
780 filename: path to file to be parsed.
781 Returns:
782 A list of dictionaris, each of them includes the name,
783 the type, and the default value of the target config.
784 """
785 config_attrs = []
786 linenum = 1
787
788 for line in open(filename):
789 tokens = line.split()
790 if len(tokens) != 3:
791 bad_recipe(filename, linenum,
792 "%d fields in this line. Each line must contain 3 fields"
793 % len(tokens))
794
795 (config, type, default) = tokens
796
797 # prefix the option name with CONFIG_ if missing
798 if not config.startswith('CONFIG_'):
799 config = 'CONFIG_' + config
800
801 # sanity check of default values
802 if type == 'bool':
803 if not default in ('y', 'n'):
804 bad_recipe(filename, linenum,
805 "default for bool type must be either y or n")
806 elif type == 'tristate':
807 if not default in ('y', 'm', 'n'):
808 bad_recipe(filename, linenum,
809 "default for tristate type must be y, m, or n")
810 elif type == 'string':
811 if default[0] != '"' or default[-1] != '"':
812 bad_recipe(filename, linenum,
813 "default for string type must be surrounded by double-quotations")
814 elif type == 'int':
815 try:
816 int(default)
817 except:
818 bad_recipe(filename, linenum,
819 "type is int, but default value is not decimal")
820 elif type == 'hex':
821 if len(default) < 2 or default[:2] != '0x':
822 bad_recipe(filename, linenum,
823 "default for hex type must be prefixed with 0x")
824 try:
825 int(default, 16)
826 except:
827 bad_recipe(filename, linenum,
828 "type is hex, but default value is not hexadecimal")
829 else:
830 bad_recipe(filename, linenum,
831 "unsupported type '%s'. type must be one of bool, tristate, string, int, hex"
832 % type)
833
834 config_attrs.append({'config': config, 'type': type, 'default': default})
835 linenum += 1
836
837 return config_attrs
838
839def main():
840 try:
841 cpu_count = multiprocessing.cpu_count()
842 except NotImplementedError:
843 cpu_count = 1
844
845 parser = optparse.OptionParser()
846 # Add options here
847 parser.add_option('-c', '--color', action='store_true', default=False,
848 help='display the log in color')
Joe Hershberger91040e82015-05-19 13:21:19 -0500849 parser.add_option('-d', '--defconfigs', type='string',
850 help='a file containing a list of defconfigs to move')
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900851 parser.add_option('-n', '--dry-run', action='store_true', default=False,
852 help='perform a trial run (show log with no changes)')
853 parser.add_option('-e', '--exit-on-error', action='store_true',
854 default=False,
855 help='exit immediately on any error')
Joe Hershberger2144f882015-05-19 13:21:20 -0500856 parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
857 action='store_true', default=False,
858 help='only cleanup the headers')
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900859 parser.add_option('-j', '--jobs', type='int', default=cpu_count,
860 help='the number of jobs to run simultaneously')
861 parser.usage += ' recipe_file\n\n' + \
862 'The recipe_file should describe config options you want to move.\n' + \
863 'Each line should contain config_name, type, default_value\n\n' + \
864 'Example:\n' + \
865 'CONFIG_FOO bool n\n' + \
866 'CONFIG_BAR int 100\n' + \
867 'CONFIG_BAZ string "hello"\n'
868
869 (options, args) = parser.parse_args()
870
871 if len(args) != 1:
872 parser.print_usage()
873 sys.exit(1)
874
875 config_attrs = parse_recipe(args[0])
876
877 update_cross_compile()
878
Joe Hershberger2144f882015-05-19 13:21:20 -0500879 check_top_directory()
880
881 if not options.cleanup_headers_only:
882 move_config(config_attrs, options)
883
884 cleanup_headers(config_attrs, options.dry_run)
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900885
886if __name__ == '__main__':
887 main()