blob: 2b2df8e7e55202dde96f98d906b6e5574f56c328 [file] [log] [blame]
Masahiro Yamada5a27c732015-05-20 11:36:07 +09001#!/usr/bin/env python2
2#
3# Author: Masahiro Yamada <yamada.masahiro@socionext.com>
4#
5# SPDX-License-Identifier: GPL-2.0+
6#
7
8"""
9Move config options from headers to defconfig files.
10
11Since Kconfig was introduced to U-Boot, we have worked on moving
12config options from headers to Kconfig (defconfig).
13
14This tool intends to help this tremendous work.
15
16
17Usage
18-----
19
20This tool takes one input file. (let's say 'recipe' file here.)
21The recipe describes the list of config options you want to move.
22Each line takes the form:
23<config_name> <type> <default>
24(the fields must be separated with whitespaces.)
25
26<config_name> is the name of config option.
27
28<type> is the type of the option. It must be one of bool, tristate,
29string, int, and hex.
30
31<default> is the default value of the option. It must be appropriate
32value corresponding to the option type. It must be either y or n for
33the bool type. Tristate options can also take m (although U-Boot has
34not supported the module feature).
35
36You can add two or more lines in the recipe file, so you can move
37multiple options at once.
38
39Let's say, for example, you want to move CONFIG_CMD_USB and
40CONFIG_SYS_TEXT_BASE.
41
42The type should be bool, hex, respectively. So, the recipe file
43should look like this:
44
45 $ cat recipe
46 CONFIG_CMD_USB bool n
47 CONFIG_SYS_TEXT_BASE hex 0x00000000
48
Joe Hershberger96464ba2015-05-19 13:21:17 -050049Next you must edit the Kconfig to add the menu entries for the configs
50you are moving.
51
Masahiro Yamada5a27c732015-05-20 11:36:07 +090052And then run this tool giving the file name of the recipe
53
54 $ tools/moveconfig.py recipe
55
56The tool walks through all the defconfig files to move the config
57options specified by the recipe file.
58
59The log is also displayed on the terminal.
60
61Each line is printed in the format
62<defconfig_name> : <action>
63
64<defconfig_name> is the name of the defconfig
65(without the suffix _defconfig).
66
67<action> shows what the tool did for that defconfig.
68It looks like one of the followings:
69
70 - Move 'CONFIG_... '
71 This config option was moved to the defconfig
72
73 - Default value 'CONFIG_...'. Do nothing.
74 The value of this option is the same as default.
75 We do not have to add it to the defconfig.
76
77 - 'CONFIG_...' already exists in Kconfig. Do nothing.
78 This config option is already defined in Kconfig.
79 We do not need/want to touch it.
80
81 - Undefined. Do nothing.
82 This config option was not found in the config header.
83 Nothing to do.
84
85 - Failed to process. Skip.
86 An error occurred during processing this defconfig. Skipped.
87 (If -e option is passed, the tool exits immediately on error.)
88
89Finally, you will be asked, Clean up headers? [y/n]:
90
91If you say 'y' here, the unnecessary config defines are removed
92from the config headers (include/configs/*.h).
93It just uses the regex method, so you should not rely on it.
94Just in case, please do 'git diff' to see what happened.
95
96
97How does it works?
98------------------
99
100This tool runs configuration and builds include/autoconf.mk for every
101defconfig. The config options defined in Kconfig appear in the .config
102file (unless they are hidden because of unmet dependency.)
103On the other hand, the config options defined by board headers are seen
104in include/autoconf.mk. The tool looks for the specified options in both
105of them to decide the appropriate action for the options. If the option
106is found in the .config or the value is the same as the specified default,
107the option does not need to be touched. If the option is found in
108include/autoconf.mk, but not in the .config, and the value is different
109from the default, the tools adds the option to the defconfig.
110
111For faster processing, this tool handles multi-threading. It creates
112separate build directories where the out-of-tree build is run. The
113temporary build directories are automatically created and deleted as
114needed. The number of threads are chosen based on the number of the CPU
115cores of your system although you can change it via -j (--jobs) option.
116
117
118Toolchains
119----------
120
121Appropriate toolchain are necessary to generate include/autoconf.mk
122for all the architectures supported by U-Boot. Most of them are available
123at the kernel.org site, some are not provided by kernel.org.
124
125The default per-arch CROSS_COMPILE used by this tool is specified by
126the list below, CROSS_COMPILE. You may wish to update the list to
127use your own. Instead of modifying the list directly, you can give
128them via environments.
129
130
131Available options
132-----------------
133
134 -c, --color
135 Surround each portion of the log with escape sequences to display it
136 in color on the terminal.
137
Joe Hershberger91040e82015-05-19 13:21:19 -0500138 -d, --defconfigs
139 Specify a file containing a list of defconfigs to move
140
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900141 -n, --dry-run
142 Peform a trial run that does not make any changes. It is useful to
143 see what is going to happen before one actually runs it.
144
145 -e, --exit-on-error
146 Exit immediately if Make exits with a non-zero status while processing
147 a defconfig file.
148
Joe Hershberger2144f882015-05-19 13:21:20 -0500149 -H, --headers-only
150 Only cleanup the headers; skip the defconfig processing
151
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900152 -j, --jobs
153 Specify the number of threads to run simultaneously. If not specified,
154 the number of threads is the same as the number of CPU cores.
155
156To see the complete list of supported options, run
157
158 $ tools/moveconfig.py -h
159
160"""
161
162import fnmatch
163import multiprocessing
164import optparse
165import os
166import re
167import shutil
168import subprocess
169import sys
170import tempfile
171import time
172
173SHOW_GNU_MAKE = 'scripts/show-gnu-make'
174SLEEP_TIME=0.03
175
176# Here is the list of cross-tools I use.
177# Most of them are available at kernel.org
178# (https://www.kernel.org/pub/tools/crosstool/files/bin/), except the followings:
179# arc: https://github.com/foss-for-synopsys-dwc-arc-processors/toolchain/releases
180# blackfin: http://sourceforge.net/projects/adi-toolchain/files/
181# nds32: http://osdk.andestech.com/packages/
182# nios2: https://sourcery.mentor.com/GNUToolchain/subscription42545
183# sh: http://sourcery.mentor.com/public/gnu_toolchain/sh-linux-gnu
184CROSS_COMPILE = {
185 'arc': 'arc-linux-',
186 'aarch64': 'aarch64-linux-',
187 'arm': 'arm-unknown-linux-gnueabi-',
188 'avr32': 'avr32-linux-',
189 'blackfin': 'bfin-elf-',
190 'm68k': 'm68k-linux-',
191 'microblaze': 'microblaze-linux-',
192 'mips': 'mips-linux-',
193 'nds32': 'nds32le-linux-',
194 'nios2': 'nios2-linux-gnu-',
195 'openrisc': 'or32-linux-',
196 'powerpc': 'powerpc-linux-',
197 'sh': 'sh-linux-gnu-',
198 'sparc': 'sparc-linux-',
199 'x86': 'i386-linux-'
200}
201
202STATE_IDLE = 0
203STATE_DEFCONFIG = 1
204STATE_AUTOCONF = 2
Joe Hershberger96464ba2015-05-19 13:21:17 -0500205STATE_SAVEDEFCONFIG = 3
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900206
207ACTION_MOVE = 0
208ACTION_DEFAULT_VALUE = 1
209ACTION_ALREADY_EXIST = 2
210ACTION_UNDEFINED = 3
211
212COLOR_BLACK = '0;30'
213COLOR_RED = '0;31'
214COLOR_GREEN = '0;32'
215COLOR_BROWN = '0;33'
216COLOR_BLUE = '0;34'
217COLOR_PURPLE = '0;35'
218COLOR_CYAN = '0;36'
219COLOR_LIGHT_GRAY = '0;37'
220COLOR_DARK_GRAY = '1;30'
221COLOR_LIGHT_RED = '1;31'
222COLOR_LIGHT_GREEN = '1;32'
223COLOR_YELLOW = '1;33'
224COLOR_LIGHT_BLUE = '1;34'
225COLOR_LIGHT_PURPLE = '1;35'
226COLOR_LIGHT_CYAN = '1;36'
227COLOR_WHITE = '1;37'
228
229### helper functions ###
230def get_devnull():
231 """Get the file object of '/dev/null' device."""
232 try:
233 devnull = subprocess.DEVNULL # py3k
234 except AttributeError:
235 devnull = open(os.devnull, 'wb')
236 return devnull
237
238def check_top_directory():
239 """Exit if we are not at the top of source directory."""
240 for f in ('README', 'Licenses'):
241 if not os.path.exists(f):
242 sys.exit('Please run at the top of source directory.')
243
244def get_make_cmd():
245 """Get the command name of GNU Make.
246
247 U-Boot needs GNU Make for building, but the command name is not
248 necessarily "make". (for example, "gmake" on FreeBSD).
249 Returns the most appropriate command name on your system.
250 """
251 process = subprocess.Popen([SHOW_GNU_MAKE], stdout=subprocess.PIPE)
252 ret = process.communicate()
253 if process.returncode:
254 sys.exit('GNU Make not found')
255 return ret[0].rstrip()
256
257def color_text(color_enabled, color, string):
258 """Return colored string."""
259 if color_enabled:
260 return '\033[' + color + 'm' + string + '\033[0m'
261 else:
262 return string
263
264def log_msg(color_enabled, color, defconfig, msg):
265 """Return the formated line for the log."""
266 return defconfig[:-len('_defconfig')].ljust(37) + ': ' + \
267 color_text(color_enabled, color, msg) + '\n'
268
269def update_cross_compile():
270 """Update per-arch CROSS_COMPILE via enviroment variables
271
272 The default CROSS_COMPILE values are available
273 in the CROSS_COMPILE list above.
274
275 You can override them via enviroment variables
276 CROSS_COMPILE_{ARCH}.
277
278 For example, if you want to override toolchain prefixes
279 for ARM and PowerPC, you can do as follows in your shell:
280
281 export CROSS_COMPILE_ARM=...
282 export CROSS_COMPILE_POWERPC=...
283 """
284 archs = []
285
286 for arch in os.listdir('arch'):
287 if os.path.exists(os.path.join('arch', arch, 'Makefile')):
288 archs.append(arch)
289
290 # arm64 is a special case
291 archs.append('aarch64')
292
293 for arch in archs:
294 env = 'CROSS_COMPILE_' + arch.upper()
295 cross_compile = os.environ.get(env)
296 if cross_compile:
297 CROSS_COMPILE[arch] = cross_compile
298
299def cleanup_one_header(header_path, patterns, dry_run):
300 """Clean regex-matched lines away from a file.
301
302 Arguments:
303 header_path: path to the cleaned file.
304 patterns: list of regex patterns. Any lines matching to these
305 patterns are deleted.
306 dry_run: make no changes, but still display log.
307 """
308 with open(header_path) as f:
309 lines = f.readlines()
310
311 matched = []
312 for i, line in enumerate(lines):
313 for pattern in patterns:
314 m = pattern.search(line)
315 if m:
316 print '%s: %s: %s' % (header_path, i + 1, line),
317 matched.append(i)
318 break
319
320 if dry_run or not matched:
321 return
322
323 with open(header_path, 'w') as f:
324 for i, line in enumerate(lines):
325 if not i in matched:
326 f.write(line)
327
328def cleanup_headers(config_attrs, dry_run):
329 """Delete config defines from board headers.
330
331 Arguments:
332 config_attrs: A list of dictionaris, each of them includes the name,
333 the type, and the default value of the target config.
334 dry_run: make no changes, but still display log.
335 """
336 while True:
337 choice = raw_input('Clean up headers? [y/n]: ').lower()
338 print choice
339 if choice == 'y' or choice == 'n':
340 break
341
342 if choice == 'n':
343 return
344
345 patterns = []
346 for config_attr in config_attrs:
347 config = config_attr['config']
348 patterns.append(re.compile(r'#\s*define\s+%s\W' % config))
349 patterns.append(re.compile(r'#\s*undef\s+%s\W' % config))
350
Joe Hershberger60727f52015-05-19 13:21:21 -0500351 for dir in 'include', 'arch', 'board':
352 for (dirpath, dirnames, filenames) in os.walk(dir):
353 for filename in filenames:
354 if not fnmatch.fnmatch(filename, '*~'):
355 cleanup_one_header(os.path.join(dirpath, filename),
356 patterns, dry_run)
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900357
358### classes ###
359class KconfigParser:
360
361 """A parser of .config and include/autoconf.mk."""
362
363 re_arch = re.compile(r'CONFIG_SYS_ARCH="(.*)"')
364 re_cpu = re.compile(r'CONFIG_SYS_CPU="(.*)"')
365
366 def __init__(self, config_attrs, options, build_dir):
367 """Create a new parser.
368
369 Arguments:
370 config_attrs: A list of dictionaris, each of them includes the name,
371 the type, and the default value of the target config.
372 options: option flags.
373 build_dir: Build directory.
374 """
375 self.config_attrs = config_attrs
376 self.options = options
377 self.build_dir = build_dir
378
379 def get_cross_compile(self):
380 """Parse .config file and return CROSS_COMPILE.
381
382 Returns:
383 A string storing the compiler prefix for the architecture.
384 """
385 arch = ''
386 cpu = ''
387 dotconfig = os.path.join(self.build_dir, '.config')
388 for line in open(dotconfig):
389 m = self.re_arch.match(line)
390 if m:
391 arch = m.group(1)
392 continue
393 m = self.re_cpu.match(line)
394 if m:
395 cpu = m.group(1)
396
397 assert arch, 'Error: arch is not defined in %s' % defconfig
398
399 # fix-up for aarch64
400 if arch == 'arm' and cpu == 'armv8':
401 arch = 'aarch64'
402
403 return CROSS_COMPILE.get(arch, '')
404
Joe Hershberger96464ba2015-05-19 13:21:17 -0500405 def parse_one_config(self, config_attr, defconfig_lines, autoconf_lines):
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900406 """Parse .config, defconfig, include/autoconf.mk for one config.
407
408 This function looks for the config options in the lines from
409 defconfig, .config, and include/autoconf.mk in order to decide
410 which action should be taken for this defconfig.
411
412 Arguments:
413 config_attr: A dictionary including the name, the type,
414 and the default value of the target config.
415 defconfig_lines: lines from the original defconfig file.
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900416 autoconf_lines: lines from the include/autoconf.mk file.
417
418 Returns:
419 A tupple of the action for this defconfig and the line
420 matched for the config.
421 """
422 config = config_attr['config']
423 not_set = '# %s is not set' % config
424
425 if config_attr['type'] in ('bool', 'tristate') and \
426 config_attr['default'] == 'n':
427 default = not_set
428 else:
429 default = config + '=' + config_attr['default']
430
Joe Hershberger96464ba2015-05-19 13:21:17 -0500431 for line in defconfig_lines:
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900432 line = line.rstrip()
433 if line.startswith(config + '=') or line == not_set:
434 return (ACTION_ALREADY_EXIST, line)
435
436 if config_attr['type'] in ('bool', 'tristate'):
437 value = not_set
438 else:
439 value = '(undefined)'
440
441 for line in autoconf_lines:
442 line = line.rstrip()
443 if line.startswith(config + '='):
444 value = line
445 break
446
447 if value == default:
448 action = ACTION_DEFAULT_VALUE
449 elif value == '(undefined)':
450 action = ACTION_UNDEFINED
451 else:
452 action = ACTION_MOVE
453
454 return (action, value)
455
456 def update_defconfig(self, defconfig):
457 """Parse files for the config options and update the defconfig.
458
459 This function parses the given defconfig, the generated .config
460 and include/autoconf.mk searching the target options.
461 Move the config option(s) to the defconfig or do nothing if unneeded.
462 Also, display the log to show what happened to this defconfig.
463
464 Arguments:
465 defconfig: defconfig name.
466 """
467
468 defconfig_path = os.path.join('configs', defconfig)
469 dotconfig_path = os.path.join(self.build_dir, '.config')
470 autoconf_path = os.path.join(self.build_dir, 'include', 'autoconf.mk')
471 results = []
472
473 with open(defconfig_path) as f:
474 defconfig_lines = f.readlines()
475
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900476 with open(autoconf_path) as f:
477 autoconf_lines = f.readlines()
478
479 for config_attr in self.config_attrs:
480 result = self.parse_one_config(config_attr, defconfig_lines,
Joe Hershberger96464ba2015-05-19 13:21:17 -0500481 autoconf_lines)
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900482 results.append(result)
483
484 log = ''
485
486 for (action, value) in results:
487 if action == ACTION_MOVE:
488 actlog = "Move '%s'" % value
489 log_color = COLOR_LIGHT_GREEN
490 elif action == ACTION_DEFAULT_VALUE:
491 actlog = "Default value '%s'. Do nothing." % value
492 log_color = COLOR_LIGHT_BLUE
493 elif action == ACTION_ALREADY_EXIST:
494 actlog = "'%s' already defined in Kconfig. Do nothing." % value
495 log_color = COLOR_LIGHT_PURPLE
496 elif action == ACTION_UNDEFINED:
497 actlog = "Undefined. Do nothing."
498 log_color = COLOR_DARK_GRAY
499 else:
500 sys.exit("Internal Error. This should not happen.")
501
502 log += log_msg(self.options.color, log_color, defconfig, actlog)
503
504 # Some threads are running in parallel.
505 # Print log in one shot to not mix up logs from different threads.
506 print log,
507
508 if not self.options.dry_run:
Joe Hershberger96464ba2015-05-19 13:21:17 -0500509 with open(dotconfig_path, 'a') as f:
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900510 for (action, value) in results:
511 if action == ACTION_MOVE:
512 f.write(value + '\n')
513
514 os.remove(os.path.join(self.build_dir, 'include', 'config', 'auto.conf'))
515 os.remove(autoconf_path)
516
517class Slot:
518
519 """A slot to store a subprocess.
520
521 Each instance of this class handles one subprocess.
522 This class is useful to control multiple threads
523 for faster processing.
524 """
525
526 def __init__(self, config_attrs, options, devnull, make_cmd):
527 """Create a new process slot.
528
529 Arguments:
530 config_attrs: A list of dictionaris, each of them includes the name,
531 the type, and the default value of the target config.
532 options: option flags.
533 devnull: A file object of '/dev/null'.
534 make_cmd: command name of GNU Make.
535 """
536 self.options = options
537 self.build_dir = tempfile.mkdtemp()
538 self.devnull = devnull
539 self.make_cmd = (make_cmd, 'O=' + self.build_dir)
540 self.parser = KconfigParser(config_attrs, options, self.build_dir)
541 self.state = STATE_IDLE
542 self.failed_boards = []
543
544 def __del__(self):
545 """Delete the working directory
546
547 This function makes sure the temporary directory is cleaned away
548 even if Python suddenly dies due to error. It should be done in here
549 because it is guranteed the destructor is always invoked when the
550 instance of the class gets unreferenced.
551
552 If the subprocess is still running, wait until it finishes.
553 """
554 if self.state != STATE_IDLE:
555 while self.ps.poll() == None:
556 pass
557 shutil.rmtree(self.build_dir)
558
559 def add(self, defconfig):
560 """Assign a new subprocess for defconfig and add it to the slot.
561
562 If the slot is vacant, create a new subprocess for processing the
563 given defconfig and add it to the slot. Just returns False if
564 the slot is occupied (i.e. the current subprocess is still running).
565
566 Arguments:
567 defconfig: defconfig name.
568
569 Returns:
570 Return True on success or False on failure
571 """
572 if self.state != STATE_IDLE:
573 return False
574 cmd = list(self.make_cmd)
575 cmd.append(defconfig)
576 self.ps = subprocess.Popen(cmd, stdout=self.devnull)
577 self.defconfig = defconfig
578 self.state = STATE_DEFCONFIG
579 return True
580
581 def poll(self):
582 """Check the status of the subprocess and handle it as needed.
583
584 Returns True if the slot is vacant (i.e. in idle state).
585 If the configuration is successfully finished, assign a new
586 subprocess to build include/autoconf.mk.
587 If include/autoconf.mk is generated, invoke the parser to
588 parse the .config and the include/autoconf.mk, and then set the
589 slot back to the idle state.
590
591 Returns:
592 Return True if the subprocess is terminated, False otherwise
593 """
594 if self.state == STATE_IDLE:
595 return True
596
597 if self.ps.poll() == None:
598 return False
599
600 if self.ps.poll() != 0:
601
602 print >> sys.stderr, log_msg(self.options.color,
603 COLOR_LIGHT_RED,
604 self.defconfig,
605 "failed to process.")
606 if self.options.exit_on_error:
607 sys.exit("Exit on error.")
608 else:
609 # If --exit-on-error flag is not set,
610 # skip this board and continue.
611 # Record the failed board.
612 self.failed_boards.append(self.defconfig)
613 self.state = STATE_IDLE
614 return True
615
616 if self.state == STATE_AUTOCONF:
617 self.parser.update_defconfig(self.defconfig)
Joe Hershberger96464ba2015-05-19 13:21:17 -0500618
619 """Save off the defconfig in a consistent way"""
620 cmd = list(self.make_cmd)
621 cmd.append('savedefconfig')
622 self.ps = subprocess.Popen(cmd, stdout=self.devnull,
623 stderr=self.devnull)
624 self.state = STATE_SAVEDEFCONFIG
625 return False
626
627 if self.state == STATE_SAVEDEFCONFIG:
628 defconfig_path = os.path.join(self.build_dir, 'defconfig')
629 shutil.move(defconfig_path,
630 os.path.join('configs', self.defconfig))
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900631 self.state = STATE_IDLE
632 return True
633
634 cross_compile = self.parser.get_cross_compile()
635 cmd = list(self.make_cmd)
636 if cross_compile:
637 cmd.append('CROSS_COMPILE=%s' % cross_compile)
Joe Hershberger7740f652015-05-19 13:21:18 -0500638 cmd.append('KCONFIG_IGNORE_DUPLICATES=1')
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900639 cmd.append('include/config/auto.conf')
640 self.ps = subprocess.Popen(cmd, stdout=self.devnull)
641 self.state = STATE_AUTOCONF
642 return False
643
644 def get_failed_boards(self):
645 """Returns a list of failed boards (defconfigs) in this slot.
646 """
647 return self.failed_boards
648
649class Slots:
650
651 """Controller of the array of subprocess slots."""
652
653 def __init__(self, config_attrs, options):
654 """Create a new slots controller.
655
656 Arguments:
657 config_attrs: A list of dictionaris containing the name, the type,
658 and the default value of the target CONFIG.
659 options: option flags.
660 """
661 self.options = options
662 self.slots = []
663 devnull = get_devnull()
664 make_cmd = get_make_cmd()
665 for i in range(options.jobs):
666 self.slots.append(Slot(config_attrs, options, devnull, make_cmd))
667
668 def add(self, defconfig):
669 """Add a new subprocess if a vacant slot is found.
670
671 Arguments:
672 defconfig: defconfig name to be put into.
673
674 Returns:
675 Return True on success or False on failure
676 """
677 for slot in self.slots:
678 if slot.add(defconfig):
679 return True
680 return False
681
682 def available(self):
683 """Check if there is a vacant slot.
684
685 Returns:
686 Return True if at lease one vacant slot is found, False otherwise.
687 """
688 for slot in self.slots:
689 if slot.poll():
690 return True
691 return False
692
693 def empty(self):
694 """Check if all slots are vacant.
695
696 Returns:
697 Return True if all the slots are vacant, False otherwise.
698 """
699 ret = True
700 for slot in self.slots:
701 if not slot.poll():
702 ret = False
703 return ret
704
705 def show_failed_boards(self):
706 """Display all of the failed boards (defconfigs)."""
707 failed_boards = []
708
709 for slot in self.slots:
710 failed_boards += slot.get_failed_boards()
711
712 if len(failed_boards) > 0:
713 msg = [ "The following boards were not processed due to error:" ]
714 msg += failed_boards
715 for line in msg:
716 print >> sys.stderr, color_text(self.options.color,
717 COLOR_LIGHT_RED, line)
718
Joe Hershberger2559cd82015-05-19 13:21:22 -0500719 with open('moveconfig.failed', 'w') as f:
720 for board in failed_boards:
721 f.write(board + '\n')
722
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900723def move_config(config_attrs, options):
724 """Move config options to defconfig files.
725
726 Arguments:
727 config_attrs: A list of dictionaris, each of them includes the name,
728 the type, and the default value of the target config.
729 options: option flags
730 """
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900731 if len(config_attrs) == 0:
732 print 'Nothing to do. exit.'
733 sys.exit(0)
734
735 print 'Move the following CONFIG options (jobs: %d)' % options.jobs
736 for config_attr in config_attrs:
737 print ' %s (type: %s, default: %s)' % (config_attr['config'],
738 config_attr['type'],
739 config_attr['default'])
740
Joe Hershberger91040e82015-05-19 13:21:19 -0500741 if options.defconfigs:
742 defconfigs = [line.strip() for line in open(options.defconfigs)]
743 for i, defconfig in enumerate(defconfigs):
744 if not defconfig.endswith('_defconfig'):
745 defconfigs[i] = defconfig + '_defconfig'
746 if not os.path.exists(os.path.join('configs', defconfigs[i])):
747 sys.exit('%s - defconfig does not exist. Stopping.' %
748 defconfigs[i])
749 else:
750 # All the defconfig files to be processed
751 defconfigs = []
752 for (dirpath, dirnames, filenames) in os.walk('configs'):
753 dirpath = dirpath[len('configs') + 1:]
754 for filename in fnmatch.filter(filenames, '*_defconfig'):
755 defconfigs.append(os.path.join(dirpath, filename))
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900756
757 slots = Slots(config_attrs, options)
758
759 # Main loop to process defconfig files:
760 # Add a new subprocess into a vacant slot.
761 # Sleep if there is no available slot.
762 for defconfig in defconfigs:
763 while not slots.add(defconfig):
764 while not slots.available():
765 # No available slot: sleep for a while
766 time.sleep(SLEEP_TIME)
767
768 # wait until all the subprocesses finish
769 while not slots.empty():
770 time.sleep(SLEEP_TIME)
771
772 slots.show_failed_boards()
773
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900774def bad_recipe(filename, linenum, msg):
775 """Print error message with the file name and the line number and exit."""
776 sys.exit("%s: line %d: error : " % (filename, linenum) + msg)
777
778def parse_recipe(filename):
779 """Parse the recipe file and retrieve the config attributes.
780
781 This function parses the given recipe file and gets the name,
782 the type, and the default value of the target config options.
783
784 Arguments:
785 filename: path to file to be parsed.
786 Returns:
787 A list of dictionaris, each of them includes the name,
788 the type, and the default value of the target config.
789 """
790 config_attrs = []
791 linenum = 1
792
793 for line in open(filename):
794 tokens = line.split()
795 if len(tokens) != 3:
796 bad_recipe(filename, linenum,
797 "%d fields in this line. Each line must contain 3 fields"
798 % len(tokens))
799
800 (config, type, default) = tokens
801
802 # prefix the option name with CONFIG_ if missing
803 if not config.startswith('CONFIG_'):
804 config = 'CONFIG_' + config
805
806 # sanity check of default values
807 if type == 'bool':
808 if not default in ('y', 'n'):
809 bad_recipe(filename, linenum,
810 "default for bool type must be either y or n")
811 elif type == 'tristate':
812 if not default in ('y', 'm', 'n'):
813 bad_recipe(filename, linenum,
814 "default for tristate type must be y, m, or n")
815 elif type == 'string':
816 if default[0] != '"' or default[-1] != '"':
817 bad_recipe(filename, linenum,
818 "default for string type must be surrounded by double-quotations")
819 elif type == 'int':
820 try:
821 int(default)
822 except:
823 bad_recipe(filename, linenum,
824 "type is int, but default value is not decimal")
825 elif type == 'hex':
826 if len(default) < 2 or default[:2] != '0x':
827 bad_recipe(filename, linenum,
828 "default for hex type must be prefixed with 0x")
829 try:
830 int(default, 16)
831 except:
832 bad_recipe(filename, linenum,
833 "type is hex, but default value is not hexadecimal")
834 else:
835 bad_recipe(filename, linenum,
836 "unsupported type '%s'. type must be one of bool, tristate, string, int, hex"
837 % type)
838
839 config_attrs.append({'config': config, 'type': type, 'default': default})
840 linenum += 1
841
842 return config_attrs
843
844def main():
845 try:
846 cpu_count = multiprocessing.cpu_count()
847 except NotImplementedError:
848 cpu_count = 1
849
850 parser = optparse.OptionParser()
851 # Add options here
852 parser.add_option('-c', '--color', action='store_true', default=False,
853 help='display the log in color')
Joe Hershberger91040e82015-05-19 13:21:19 -0500854 parser.add_option('-d', '--defconfigs', type='string',
855 help='a file containing a list of defconfigs to move')
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900856 parser.add_option('-n', '--dry-run', action='store_true', default=False,
857 help='perform a trial run (show log with no changes)')
858 parser.add_option('-e', '--exit-on-error', action='store_true',
859 default=False,
860 help='exit immediately on any error')
Joe Hershberger2144f882015-05-19 13:21:20 -0500861 parser.add_option('-H', '--headers-only', dest='cleanup_headers_only',
862 action='store_true', default=False,
863 help='only cleanup the headers')
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900864 parser.add_option('-j', '--jobs', type='int', default=cpu_count,
865 help='the number of jobs to run simultaneously')
866 parser.usage += ' recipe_file\n\n' + \
867 'The recipe_file should describe config options you want to move.\n' + \
868 'Each line should contain config_name, type, default_value\n\n' + \
869 'Example:\n' + \
870 'CONFIG_FOO bool n\n' + \
871 'CONFIG_BAR int 100\n' + \
872 'CONFIG_BAZ string "hello"\n'
873
874 (options, args) = parser.parse_args()
875
876 if len(args) != 1:
877 parser.print_usage()
878 sys.exit(1)
879
880 config_attrs = parse_recipe(args[0])
881
882 update_cross_compile()
883
Joe Hershberger2144f882015-05-19 13:21:20 -0500884 check_top_directory()
885
886 if not options.cleanup_headers_only:
887 move_config(config_attrs, options)
888
889 cleanup_headers(config_attrs, options.dry_run)
Masahiro Yamada5a27c732015-05-20 11:36:07 +0900890
891if __name__ == '__main__':
892 main()