drashna/qmk_firmware
1
0
Fork 0
Code Activity Actions

2020 November 28 Breaking Changes Update (#11053)

Browse source 
* Branch point for 2020 November 28 Breaking Change                                                

* Remove matrix_col_t to allow MATRIX_ROWS > 32 (#10183)                                           

* Add support for soft serial to ATmega32U2 (#10204)                                               

* Change MIDI velocity implementation to allow direct control of velocity value (#9940)            

* Add ability to build a subset of all keyboards based on platform.                                

* Actually use eeprom_driver_init().                                                               

* Make bootloader_jump weak for ChibiOS. (#10417)                                                  

* Joystick 16-bit support (#10439)                                                                 

* Per-encoder resolutions (#10259)                                                                 

* Share button state from mousekey to pointing_device (#10179)                                     

* Add hotfix for chibios keyboards not wake (#10088)                                               

* Add advanced/efficient RGB Matrix Indicators (#8564)                                             

* Naming change.                                                                                   

* Support for STM32 GPIOF,G,H,I,J,K (#10206)                                                       

* Add milc as a dependency and remove the installed milc (#10563)                                  

* ChibiOS upgrade: early init conversions (#10214)                                                 

* ChibiOS upgrade: configuration file migrator (#9952)                                             

* Haptic and solenoid cleanup (#9700)                                                              

* XD75 cleanup (#10524)                                                                            

* OLED display update interval support (#10388)                                                    

* Add definition based on currently-selected serial driver. (#10716)                               

* New feature: Retro Tapping per key (#10622)                                                      

* Allow for modification of output RGB values when using rgblight/rgb_matrix. (#10638)             

* Add housekeeping task callbacks so that keyboards/keymaps are capable of executing code for each main loop iteration. (#10530)

* Rescale both ChibiOS and AVR backlighting.                                                       

* Reduce Helix keyboard build variation (#8669)                                                    

* Minor change to behavior allowing display updates to continue between task ticks (#10750)        

* Some GPIO manipulations in matrix.c change to atomic. (#10491)                                   

* qmk cformat (#10767)                                                                             

* [Keyboard] Update the Speedo firmware for v3.0 (#10657)                                          

* Maartenwut/Maarten namechange to evyd13/Evy (#10274)                                             

* [quantum] combine repeated lines of code (#10837)                                                

* Add step sequencer feature (#9703)                                                               

* aeboards/ext65 refactor (#10820)                                                                 

* Refactor xelus/dawn60 for Rev2 later (#10584)                                                    

* add DEBUG_MATRIX_SCAN_RATE_ENABLE to common_features.mk (#10824)                                 

* [Core] Added `add_oneshot_mods` & `del_oneshot_mods` (#10549)                                    

* update chibios os usb for the otg driver (#8893)                                                 

* Remove HD44780 References, Part 4 (#10735)                                                       

* [Keyboard] Add Valor FRL TKL (+refactor) (#10512)                                                

* Fix cursor position bug in oled_write_raw functions (#10800)                                     

* Fixup version.h writing when using SKIP_VERSION=yes (#10972)                                     

* Allow for certain code in the codebase assuming length of string. (#10974)                       

* Add AT90USB support for serial.c (#10706)                                                        

* Auto shift: support repeats and early registration (#9826)                                       

* Rename ledmatrix.h to match .c file (#7949)                                                      

* Split RGB_MATRIX_ENABLE into _ENABLE and _DRIVER (#10231)                                        

* Split LED_MATRIX_ENABLE into _ENABLE and _DRIVER (#10840)                                        

* Merge point for 2020 Nov 28 Breaking Change
This commit is contained in:
James Young 2020-11-28 12:02:18 -08:00 • committed by GitHub
parent 15385d4113
commit c66df16644
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
884 changed files with 8121 additions and 11685 deletions
Download patch file Download diff file Expand all files Collapse all files

826
lib/python/milc.py
View file

@ -1,826 +0,0 @@
#!/usr/bin/env python3
# coding=utf-8
"""MILC - A CLI Framework
PYTHON_ARGCOMPLETE_OK
MILC is an opinionated framework for writing CLI apps. It optimizes for the
most common unix tool pattern- small tools that are run from the command
line but generally do not feature any user interaction while they run.
For more details see the MILC documentation:
<https://github.com/clueboard/milc/tree/master/docs>
"""
from __future__ import division, print_function, unicode_literals
import argparse
import logging
import os
import re
import shlex
import subprocess
import sys
from decimal import Decimal
from pathlib import Path
from platform import platform
from tempfile import NamedTemporaryFile
from time import sleep
try:
from ConfigParser import RawConfigParser
except ImportError:
from configparser import RawConfigParser
try:
import thread
import threading
except ImportError:
thread = None
import argcomplete
import colorama
from appdirs import user_config_dir
# Disable logging until we can configure it how the user wants
logging.basicConfig(stream=os.devnull)
# Log Level Representations
EMOJI_LOGLEVELS = {
'CRITICAL': '{bg_red}{fg_white}¬_¬{style_reset_all}',
'ERROR': '{fg_red}☒{style_reset_all}',
'WARNING': '{fg_yellow}⚠{style_reset_all}',
'INFO': '{fg_blue}ℹ{style_reset_all}',
'DEBUG': '{fg_cyan}☐{style_reset_all}',
'NOTSET': '{style_reset_all}¯\\_(o_o)_/¯'
}
EMOJI_LOGLEVELS['FATAL'] = EMOJI_LOGLEVELS['CRITICAL']
EMOJI_LOGLEVELS['WARN'] = EMOJI_LOGLEVELS['WARNING']
UNICODE_SUPPORT = sys.stdout.encoding.lower().startswith('utf')
# ANSI Color setup
# Regex was gratefully borrowed from kfir on stackoverflow:
# https://stackoverflow.com/a/45448194
ansi_regex = r'\x1b(' \
r'(\[\??\d+[hl])|' \
r'([=<>a-kzNM78])|' \
r'([\(\)][a-b0-2])|' \
r'(\[\d{0,2}[ma-dgkjqi])|' \
r'(\[\d+;\d+[hfy]?)|' \
r'(\[;?[hf])|' \
r'(#[3-68])|' \
r'([01356]n)|' \
r'(O[mlnp-z]?)|' \
r'(/Z)|' \
r'(\d+)|' \
r'(\[\?\d;\d0c)|' \
r'(\d;\dR))'
ansi_escape = re.compile(ansi_regex, flags=re.IGNORECASE)
ansi_styles = (
('fg', colorama.ansi.AnsiFore()),
('bg', colorama.ansi.AnsiBack()),
('style', colorama.ansi.AnsiStyle()),
)
ansi_colors = {}
for prefix, obj in ansi_styles:
for color in [x for x in obj.__dict__ if not x.startswith('_')]:
ansi_colors[prefix + '_' + color.lower()] = getattr(obj, color)
def format_ansi(text):
"""Return a copy of text with certain strings replaced with ansi.
"""
# Avoid .format() so we don't have to worry about the log content
for color in ansi_colors:
text = text.replace('{%s}' % color, ansi_colors[color])
return text + ansi_colors['style_reset_all']
class ANSIFormatterMixin(object):
"""A log formatter mixin that inserts ANSI color.
"""
def format(self, record):
msg = super(ANSIFormatterMixin, self).format(record)
return format_ansi(msg)
class ANSIStrippingMixin(object):
"""A log formatter mixin that strips ANSI.
"""
def format(self, record):
msg = super(ANSIStrippingMixin, self).format(record)
record.levelname = ansi_escape.sub('', record.levelname)
return ansi_escape.sub('', msg)
class EmojiLoglevelMixin(object):
"""A log formatter mixin that makes the loglevel an emoji on UTF capable terminals.
"""
def format(self, record):
if UNICODE_SUPPORT:
record.levelname = EMOJI_LOGLEVELS[record.levelname].format(**ansi_colors)
return super(EmojiLoglevelMixin, self).format(record)
class ANSIFormatter(ANSIFormatterMixin, logging.Formatter):
"""A log formatter that colorizes output.
"""
pass
class ANSIStrippingFormatter(ANSIStrippingMixin, ANSIFormatterMixin, logging.Formatter):
"""A log formatter that strips ANSI
"""
pass
class ANSIEmojiLoglevelFormatter(EmojiLoglevelMixin, ANSIFormatterMixin, logging.Formatter):
"""A log formatter that adds Emoji and ANSI
"""
pass
class ANSIStrippingEmojiLoglevelFormatter(ANSIStrippingMixin, EmojiLoglevelMixin, ANSIFormatterMixin, logging.Formatter):
"""A log formatter that adds Emoji and strips ANSI
"""
pass
class Configuration(object):
"""Represents the running configuration.
This class never raises IndexError, instead it will return None if a
section or option does not yet exist.
"""
def __contains__(self, key):
return self._config.__contains__(key)
def __iter__(self):
return self._config.__iter__()
def __len__(self):
return self._config.__len__()
def __repr__(self):
return self._config.__repr__()
def keys(self):
return self._config.keys()
def items(self):
return self._config.items()
def values(self):
return self._config.values()
def __init__(self, *args, **kwargs):
self._config = {}
def __getattr__(self, key):
return self.__getitem__(key)
def __getitem__(self, key):
"""Returns a config section, creating it if it doesn't exist yet.
"""
if key not in self._config:
self.__dict__[key] = self._config[key] = ConfigurationSection(self)
return self._config[key]
def __setitem__(self, key, value):
self.__dict__[key] = value
self._config[key] = value
def __delitem__(self, key):
if key in self.__dict__ and key[0] != '_':
del self.__dict__[key]
if key in self._config:
del self._config[key]
class ConfigurationSection(Configuration):
def __init__(self, parent, *args, **kwargs):
super(ConfigurationSection, self).__init__(*args, **kwargs)
self.parent = parent
def __getitem__(self, key):
"""Returns a config value, pulling from the `user` section as a fallback.
This is called when the attribute is accessed either via the get method or through [ ] index.
"""
if key in self._config and self._config.get(key) is not None:
return self._config[key]
elif key in self.parent.user:
return self.parent.user[key]
return None
def __getattr__(self, key):
"""Returns the config value from the `user` section.
This is called when the attribute is accessed via dot notation but does not exists.
"""
if key in self.parent.user:
return self.parent.user[key]
return None
def handle_store_boolean(self, *args, **kwargs):
"""Does the add_argument for action='store_boolean'.
"""
disabled_args = None
disabled_kwargs = kwargs.copy()
disabled_kwargs['action'] = 'store_false'
disabled_kwargs['dest'] = self.get_argument_name(*args, **kwargs)
disabled_kwargs['help'] = 'Disable ' + kwargs['help']
kwargs['action'] = 'store_true'
kwargs['help'] = 'Enable ' + kwargs['help']
for flag in args:
if flag[:2] == '--':
disabled_args = ('--no-' + flag[2:],)
break
self.add_argument(*args, **kwargs)
self.add_argument(*disabled_args, **disabled_kwargs)
return (args, kwargs, disabled_args, disabled_kwargs)
class SubparserWrapper(object):
"""Wrap subparsers so we can track what options the user passed.
"""
def __init__(self, cli, submodule, subparser):
self.cli = cli
self.submodule = submodule
self.subparser = subparser
for attr in dir(subparser):
if not hasattr(self, attr):
setattr(self, attr, getattr(subparser, attr))
def completer(self, completer):
"""Add an arpcomplete completer to this subcommand.
"""
self.subparser.completer = completer
def add_argument(self, *args, **kwargs):
"""Add an argument for this subcommand.
This also stores the default for the argument in `self.cli.default_arguments`.
"""
if kwargs.get('action') == 'store_boolean':
# Store boolean will call us again with the enable/disable flag arguments
return handle_store_boolean(self, *args, **kwargs)
self.cli.acquire_lock()
argument_name = self.cli.get_argument_name(*args, **kwargs)
self.subparser.add_argument(*args, **kwargs)
if kwargs.get('action') == 'store_false':
self.cli._config_store_false.append(argument_name)
if kwargs.get('action') == 'store_true':
self.cli._config_store_true.append(argument_name)
if self.submodule not in self.cli.default_arguments:
self.cli.default_arguments[self.submodule] = {}
self.cli.default_arguments[self.submodule][argument_name] = kwargs.get('default')
self.cli.release_lock()
class MILC(object):
"""MILC - An Opinionated Batteries Included Framework
"""
def __init__(self):
"""Initialize the MILC object.
version
The version string to associate with your CLI program
"""
# Setup a lock for thread safety
self._lock = threading.RLock() if thread else None
# Define some basic info
self.acquire_lock()
self._config_store_true = []
self._config_store_false = []
self._description = None
self._entrypoint = None
self._inside_context_manager = False
self.ansi = ansi_colors
self.arg_only = {}
self.config = self.config_source = None
self.config_file = None
self.default_arguments = {}
self.version = 'unknown'
self.platform = platform()
# Figure out our program name
self.prog_name = sys.argv[0][:-3] if sys.argv[0].endswith('.py') else sys.argv[0]
self.prog_name = self.prog_name.split('/')[-1]
self.release_lock()
# Initialize all the things
self.read_config_file()
self.initialize_argparse()
self.initialize_logging()
@property
def description(self):
return self._description
@description.setter
def description(self, value):
self._description = self._arg_parser.description = value
def echo(self, text, *args, **kwargs):
"""Print colorized text to stdout.
ANSI color strings (such as {fg-blue}) will be converted into ANSI
escape sequences, and the ANSI reset sequence will be added to all
strings.
If *args or **kwargs are passed they will be used to %-format the strings.
If `self.config.general.color` is False any ANSI escape sequences in the text will be stripped.
"""
if args and kwargs:
raise RuntimeError('You can only specify *args or **kwargs, not both!')
args = args or kwargs
text = format_ansi(text)
if not self.config.general.color:
text = ansi_escape.sub('', text)
print(text % args)
def run(self, command, *args, **kwargs):
"""Run a command with subprocess.run
The *args and **kwargs arguments get passed directly to `subprocess.run`.
"""
if isinstance(command, str):
raise TypeError('`command` must be a non-text sequence such as list or tuple.')
if 'windows' in self.platform.lower():
safecmd = map(shlex.quote, command)
safecmd = ' '.join(safecmd)
command = [os.environ['SHELL'], '-c', safecmd]
self.log.debug('Running command: %s', command)
return subprocess.run(command, *args, **kwargs)
def initialize_argparse(self):
"""Prepare to process arguments from sys.argv.
"""
kwargs = {
'fromfile_prefix_chars': '@',
'conflict_handler': 'resolve',
}
self.acquire_lock()
self.subcommands = {}
self._subparsers = None
self.argwarn = argcomplete.warn
self.args = None
self._arg_parser = argparse.ArgumentParser(**kwargs)
self.set_defaults = self._arg_parser.set_defaults
self.print_usage = self._arg_parser.print_usage
self.print_help = self._arg_parser.print_help
self.release_lock()
def completer(self, completer):
"""Add an argcomplete completer to this subcommand.
"""
self._arg_parser.completer = completer
def add_argument(self, *args, **kwargs):
"""Wrapper to add arguments and track whether they were passed on the command line.
"""
if 'action' in kwargs and kwargs['action'] == 'store_boolean':
return handle_store_boolean(self, *args, **kwargs)
self.acquire_lock()
self._arg_parser.add_argument(*args, **kwargs)
if 'general' not in self.default_arguments:
self.default_arguments['general'] = {}
self.default_arguments['general'][self.get_argument_name(*args, **kwargs)] = kwargs.get('default')
self.release_lock()
def initialize_logging(self):
"""Prepare the defaults for the logging infrastructure.
"""
self.acquire_lock()
self.log_file = None
self.log_file_mode = 'a'
self.log_file_handler = None
self.log_print = True
self.log_print_to = sys.stderr
self.log_print_level = logging.INFO
self.log_file_level = logging.DEBUG
self.log_level = logging.INFO
self.log = logging.getLogger(self.__class__.__name__)
self.log.setLevel(logging.DEBUG)
logging.root.setLevel(logging.DEBUG)
self.release_lock()
self.add_argument('-V', '--version', version=self.version, action='version', help='Display the version and exit')
self.add_argument('-v', '--verbose', action='store_true', help='Make the logging more verbose')
self.add_argument('--datetime-fmt', default='%Y-%m-%d %H:%M:%S', help='Format string for datetimes')
self.add_argument('--log-fmt', default='%(levelname)s %(message)s', help='Format string for printed log output')
self.add_argument('--log-file-fmt', default='[%(levelname)s] [%(asctime)s] [file:%(pathname)s] [line:%(lineno)d] %(message)s', help='Format string for log file.')
self.add_argument('--log-file', help='File to write log messages to')
self.add_argument('--color', action='store_boolean', default=True, help='color in output')
self.add_argument('--config-file', help='The location for the configuration file')
self.arg_only['config_file'] = ['general']
def add_subparsers(self, title='Sub-commands', **kwargs):
if self._inside_context_manager:
raise RuntimeError('You must run this before the with statement!')
self.acquire_lock()
self._subparsers = self._arg_parser.add_subparsers(title=title, dest='subparsers', **kwargs)
self.release_lock()
def acquire_lock(self):
"""Acquire the MILC lock for exclusive access to properties.
"""
if self._lock:
self._lock.acquire()
def release_lock(self):
"""Release the MILC lock.
"""
if self._lock:
self._lock.release()
def find_config_file(self):
"""Locate the config file.
"""
if self.config_file:
return self.config_file
if '--config-file' in sys.argv:
return Path(sys.argv[sys.argv.index('--config-file') + 1]).expanduser().resolve()
filedir = user_config_dir(appname='qmk', appauthor='QMK')
filename = '%s.ini' % self.prog_name
return Path(filedir) / filename
def get_argument_name(self, *args, **kwargs):
"""Takes argparse arguments and returns the dest name.
"""
try:
return self._arg_parser._get_optional_kwargs(*args, **kwargs)['dest']
except ValueError:
return self._arg_parser._get_positional_kwargs(*args, **kwargs)['dest']
def argument(self, *args, **kwargs):
"""Decorator to call self.add_argument or self.<subcommand>.add_argument.
"""
if self._inside_context_manager:
raise RuntimeError('You must run this before the with statement!')
def argument_function(handler):
subcommand_name = handler.__name__.replace("_", "-")
if kwargs.get('arg_only'):
arg_name = self.get_argument_name(*args, **kwargs)
if arg_name not in self.arg_only:
self.arg_only[arg_name] = []
self.arg_only[arg_name].append(subcommand_name)
del kwargs['arg_only']
if handler is self._entrypoint:
self.add_argument(*args, **kwargs)
elif subcommand_name in self.subcommands:
self.subcommands[subcommand_name].add_argument(*args, **kwargs)
else:
raise RuntimeError('Decorated function is not entrypoint or subcommand!')
return handler
return argument_function
def arg_passed(self, arg):
"""Returns True if arg was passed on the command line.
"""
return self.default_arguments.get(arg) != self.args[arg]
def parse_args(self):
"""Parse the CLI args.
"""
if self.args:
self.log.debug('Warning: Arguments have already been parsed, ignoring duplicate attempt!')
return
argcomplete.autocomplete(self._arg_parser)
self.acquire_lock()
self.args = self._arg_parser.parse_args()
if 'entrypoint' in self.args:
self._entrypoint = self.args.entrypoint
self.release_lock()
def read_config_file(self):
"""Read in the configuration file and store it in self.config.
"""
self.acquire_lock()
self.config = Configuration()
self.config_source = Configuration()
self.config_file = self.find_config_file()
if self.config_file and self.config_file.exists():
config = RawConfigParser(self.config)
config.read(str(self.config_file))
# Iterate over the config file options and write them into self.config
for section in config.sections():
for option in config.options(section):
value = config.get(section, option)
# Coerce values into useful datatypes
if value.lower() in ['1', 'yes', 'true', 'on']:
value = True
elif value.lower() in ['0', 'no', 'false', 'off']:
value = False
elif value.lower() in ['none']:
continue
elif value.replace('.', '').isdigit():
if '.' in value:
value = Decimal(value)
else:
value = int(value)
self.config[section][option] = value
self.config_source[section][option] = 'config_file'
self.release_lock()
def merge_args_into_config(self):
"""Merge CLI arguments into self.config to create the runtime configuration.
"""
self.acquire_lock()
for argument in vars(self.args):
if argument in ('subparsers', 'entrypoint'):
continue
# Find the argument's section
# Underscores in command's names are converted to dashes during initialization.
# TODO(Erovia) Find a better solution
entrypoint_name = self._entrypoint.__name__.replace("_", "-")
if entrypoint_name in self.default_arguments and argument in self.default_arguments[entrypoint_name]:
argument_found = True
section = self._entrypoint.__name__
if argument in self.default_arguments['general']:
argument_found = True
section = 'general'
if not argument_found:
raise RuntimeError('Could not find argument in `self.default_arguments`. This should be impossible!')
exit(1)
if argument not in self.arg_only or section not in self.arg_only[argument]:
# Determine the arg value and source
arg_value = getattr(self.args, argument)
if argument in self._config_store_true and arg_value:
passed_on_cmdline = True
elif argument in self._config_store_false and not arg_value:
passed_on_cmdline = True
elif arg_value is not None:
passed_on_cmdline = True
else:
passed_on_cmdline = False
# Merge this argument into self.config
if passed_on_cmdline and (argument in self.default_arguments['general'] or argument in self.default_arguments[entrypoint_name] or argument not in self.config[entrypoint_name]):
self.config[section][argument] = arg_value
self.config_source[section][argument] = 'argument'
self.release_lock()
def save_config(self):
"""Save the current configuration to the config file.
"""
self.log.debug("Saving config file to '%s'", str(self.config_file))
if not self.config_file:
self.log.warning('%s.config_file file not set, not saving config!', self.__class__.__name__)
return
self.acquire_lock()
# Generate a sanitized version of our running configuration
config = RawConfigParser()
for section_name, section in self.config._config.items():
config.add_section(section_name)
for option_name, value in section.items():
if section_name == 'general':
if option_name in ['config_file']:
continue
if value is not None:
config.set(section_name, option_name, str(value))
# Write out the config file
config_dir = self.config_file.parent
if not config_dir.exists():
config_dir.mkdir(parents=True, exist_ok=True)
with NamedTemporaryFile(mode='w', dir=str(config_dir), delete=False) as tmpfile:
config.write(tmpfile)
# Move the new config file into place atomically
if os.path.getsize(tmpfile.name) > 0:
os.replace(tmpfile.name, str(self.config_file))
else:
self.log.warning('Config file saving failed, not replacing %s with %s.', str(self.config_file), tmpfile.name)
# Housekeeping
self.release_lock()
cli.log.info('Wrote configuration to %s', shlex.quote(str(self.config_file)))
def __call__(self):
"""Execute the entrypoint function.
"""
if not self._inside_context_manager:
# If they didn't use the context manager use it ourselves
with self:
return self.__call__()
if not self._entrypoint:
raise RuntimeError('No entrypoint provided!')
return self._entrypoint(self)
def entrypoint(self, description):
"""Set the entrypoint for when no subcommand is provided.
"""
if self._inside_context_manager:
raise RuntimeError('You must run this before cli()!')
self.acquire_lock()
self.description = description
self.release_lock()
def entrypoint_func(handler):
self.acquire_lock()
self._entrypoint = handler
self.release_lock()
return handler
return entrypoint_func
def add_subcommand(self, handler, description, name=None, hidden=False, **kwargs):
"""Register a subcommand.
If name is not provided we use `handler.__name__`.
"""
if self._inside_context_manager:
raise RuntimeError('You must run this before the with statement!')
if self._subparsers is None:
self.add_subparsers(metavar="")
if not name:
name = handler.__name__.replace("_", "-")
self.acquire_lock()
if not hidden:
self._subparsers.metavar = "{%s,%s}" % (self._subparsers.metavar[1:-1], name) if self._subparsers.metavar else "{%s%s}" % (self._subparsers.metavar[1:-1], name)
kwargs['help'] = description
self.subcommands[name] = SubparserWrapper(self, name, self._subparsers.add_parser(name, **kwargs))
self.subcommands[name].set_defaults(entrypoint=handler)
self.release_lock()
return handler
def subcommand(self, description, hidden=False, **kwargs):
"""Decorator to register a subcommand.
"""
def subcommand_function(handler):
return self.add_subcommand(handler, description, hidden=hidden, **kwargs)
return subcommand_function
def setup_logging(self):
"""Called by __enter__() to setup the logging configuration.
"""
if len(logging.root.handlers) != 0:
# MILC is the only thing that should have root log handlers
logging.root.handlers = []
self.acquire_lock()
if self.config['general']['verbose']:
self.log_print_level = logging.DEBUG
self.log_file = self.config['general']['log_file'] or self.log_file
self.log_file_format = ANSIStrippingFormatter(self.config['general']['log_file_fmt'], self.config['general']['datetime_fmt'])
self.log_format = self.config['general']['log_fmt']
if self.config.general.color:
self.log_format = ANSIEmojiLoglevelFormatter(self.config.general.log_fmt, self.config.general.datetime_fmt)
else:
self.log_format = ANSIStrippingEmojiLoglevelFormatter(self.config.general.log_fmt, self.config.general.datetime_fmt)
if self.log_file:
self.log_file_handler = logging.FileHandler(self.log_file, self.log_file_mode)
self.log_file_handler.setLevel(self.log_file_level)
self.log_file_handler.setFormatter(self.log_file_format)
logging.root.addHandler(self.log_file_handler)
if self.log_print:
self.log_print_handler = logging.StreamHandler(self.log_print_to)
self.log_print_handler.setLevel(self.log_print_level)
self.log_print_handler.setFormatter(self.log_format)
logging.root.addHandler(self.log_print_handler)
self.release_lock()
def __enter__(self):
if self._inside_context_manager:
self.log.debug('Warning: context manager was entered again. This usually means that self.__call__() was called before the with statement. You probably do not want to do that.')
return
self.acquire_lock()
self._inside_context_manager = True
self.release_lock()
colorama.init()
self.parse_args()
self.merge_args_into_config()
self.setup_logging()
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.acquire_lock()
self._inside_context_manager = False
self.release_lock()
if exc_type is not None and not isinstance(SystemExit(), exc_type):
print(exc_type)
logging.exception(exc_val)
exit(255)
cli = MILC()
if __name__ == '__main__':
@cli.argument('-c', '--comma', help='comma in output', default=True, action='store_boolean')
@cli.entrypoint('My useful CLI tool with subcommands.')
def main(cli):
comma = ',' if cli.config.general.comma else ''
cli.log.info('{bg_green}{fg_red}Hello%s World!', comma)
@cli.argument('-n', '--name', help='Name to greet', default='World')
@cli.subcommand('Description of hello subcommand here.')
def hello(cli):
comma = ',' if cli.config.general.comma else ''
cli.log.info('{fg_blue}Hello%s %s!', comma, cli.config.hello.name)
def goodbye(cli):
comma = ',' if cli.config.general.comma else ''
cli.log.info('{bg_red}Goodbye%s %s!', comma, cli.config.goodbye.name)
@cli.argument('-n', '--name', help='Name to greet', default='World')
@cli.subcommand('Think a bit before greeting the user.')
def thinking(cli):
comma = ',' if cli.config.general.comma else ''
spinner = cli.spinner(text='Just a moment...', spinner='earth')
spinner.start()
sleep(2)
spinner.stop()
with cli.spinner(text='Almost there!', spinner='moon'):
sleep(2)
cli.log.info('{fg_cyan}Hello%s %s!', comma, cli.config.thinking.name)
@cli.subcommand('Show off our ANSI colors.')
def pride(cli):
cli.echo('{bg_red} ')
cli.echo('{bg_lightred_ex} ')
cli.echo('{bg_lightyellow_ex} ')
cli.echo('{bg_green} ')
cli.echo('{bg_blue} ')
cli.echo('{bg_magenta} ')
# You can register subcommands using decorators as seen above, or using functions like like this:
cli.add_subcommand(goodbye, 'This will show up in --help output.')
cli.goodbye.add_argument('-n', '--name', help='Name to bid farewell to', default='World')
cli() # Automatically picks between main(), hello() and goodbye()

1
lib/python/qmk/cli/__init__.py
View file

@ -8,6 +8,7 @@ from milc import cli
from . import c2json
from . import cformat
from . import chibios
from . import clean
from . import compile
from . import config

1
lib/python/qmk/cli/chibios/__init__.py Normal file
View file

@ -0,0 +1 @@
from . import confmigrate

161
lib/python/qmk/cli/chibios/confmigrate.py Normal file
View file

@ -0,0 +1,161 @@
"""This script automates the copying of the default keymap into your own keymap.
"""
import re
import sys
import os
from qmk.constants import QMK_FIRMWARE
from qmk.path import normpath
from milc import cli
def eprint(*args, **kwargs):
print(*args, file=sys.stderr, **kwargs)
fileHeader = """\
/* Copyright 2020 QMK
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
/*
* This file was auto-generated by:
* `qmk chibios-confupdate -i {0} -r {1}`
*/
#pragma once
"""
def collect_defines(filepath):
with open(filepath, 'r') as f:
content = f.read()
define_search = re.compile(r'(?m)^#\s*define\s+(?:.*\\\r?\n)*.*$', re.MULTILINE)
value_search = re.compile(r'^#\s*define\s+(?P<name>[a-zA-Z0-9_]+(\([^\)]*\))?)\s*(?P<value>.*)', re.DOTALL)
define_matches = define_search.findall(content)
defines = {"keys": [], "dict": {}}
for define_match in define_matches:
value_match = value_search.search(define_match)
defines["keys"].append(value_match.group("name"))
defines["dict"][value_match.group("name")] = value_match.group("value")
return defines
def check_diffs(input_defs, reference_defs):
not_present_in_input = []
not_present_in_reference = []
to_override = []
for key in reference_defs["keys"]:
if key not in input_defs["dict"]:
not_present_in_input.append(key)
continue
for key in input_defs["keys"]:
if key not in input_defs["dict"]:
not_present_in_input.append(key)
continue
for key in input_defs["keys"]:
if key in reference_defs["keys"] and input_defs["dict"][key] != reference_defs["dict"][key]:
to_override.append((key, input_defs["dict"][key]))
return (to_override, not_present_in_input, not_present_in_reference)
def migrate_chconf_h(to_override, outfile):
print(fileHeader.format(cli.args.input.relative_to(QMK_FIRMWARE), cli.args.reference.relative_to(QMK_FIRMWARE)), file=outfile)
for override in to_override:
print("#define %s %s" % (override[0], override[1]), file=outfile)
print("", file=outfile)
print("#include_next <chconf.h>\n", file=outfile)
def migrate_halconf_h(to_override, outfile):
print(fileHeader.format(cli.args.input.relative_to(QMK_FIRMWARE), cli.args.reference.relative_to(QMK_FIRMWARE)), file=outfile)
for override in to_override:
print("#define %s %s" % (override[0], override[1]), file=outfile)
print("", file=outfile)
print("#include_next <halconf.h>\n", file=outfile)
def migrate_mcuconf_h(to_override, outfile):
print(fileHeader.format(cli.args.input.relative_to(QMK_FIRMWARE), cli.args.reference.relative_to(QMK_FIRMWARE)), file=outfile)
print("#include_next <mcuconf.h>\n", file=outfile)
for override in to_override:
print("#undef %s" % (override[0]), file=outfile)
print("#define %s %s" % (override[0], override[1]), file=outfile)
print("", file=outfile)
@cli.argument('-i', '--input', type=normpath, arg_only=True, help='Specify input config file.')
@cli.argument('-r', '--reference', type=normpath, arg_only=True, help='Specify the reference file to compare against')
@cli.argument('-o', '--overwrite', arg_only=True, action='store_true', help='Overwrites the input file during migration.')
@cli.argument('-d', '--delete', arg_only=True, action='store_true', help='If the file has no overrides, migration will delete the input file.')
@cli.subcommand('Generates a migrated ChibiOS configuration file, as a result of comparing the input against a reference')
def chibios_confmigrate(cli):
"""Generates a usable ChibiOS replacement configuration file, based on a fully-defined conf and a reference config.
"""
input_defs = collect_defines(cli.args.input)
reference_defs = collect_defines(cli.args.reference)
(to_override, not_present_in_input, not_present_in_reference) = check_diffs(input_defs, reference_defs)
if len(not_present_in_input) > 0:
eprint("Keys not in input, but present inside reference (potential manual migration required):")
for key in not_present_in_input:
eprint(" %s" % (key))
if len(not_present_in_reference) > 0:
eprint("Keys not in reference, but present inside input (potential manual migration required):")
for key in not_present_in_reference:
eprint(" %s" % (key))
if len(to_override) == 0:
eprint('No overrides found! If there were no missing keys above, it should be safe to delete the input file.')
if cli.args.delete:
os.remove(cli.args.input)
else:
eprint('Overrides found:')
for override in to_override:
eprint("%40s: %s -> %s" % (override[0], reference_defs["dict"][override[0]].encode('unicode_escape').decode("utf-8"), override[1].encode('unicode_escape').decode("utf-8")))
eprint('--------------------------------------')
if "CHCONF_H" in input_defs["dict"] or "_CHCONF_H_" in input_defs["dict"]:
migrate_chconf_h(to_override, outfile=sys.stdout)
if cli.args.overwrite:
with open(cli.args.input, "w") as out_file:
migrate_chconf_h(to_override, outfile=out_file)
elif "HALCONF_H" in input_defs["dict"] or "_HALCONF_H_" in input_defs["dict"]:
migrate_halconf_h(to_override, outfile=sys.stdout)
if cli.args.overwrite:
with open(cli.args.input, "w") as out_file:
migrate_halconf_h(to_override, outfile=out_file)
elif "MCUCONF_H" in input_defs["dict"] or "_MCUCONF_H_" in input_defs["dict"]:
migrate_mcuconf_h(to_override, outfile=sys.stdout)
if cli.args.overwrite:
with open(cli.args.input, "w") as out_file:
migrate_mcuconf_h(to_override, outfile=out_file)

2
lib/python/qmk/cli/doctor.py
View file

@ -10,9 +10,9 @@ from pathlib import Path
from enum import Enum
from milc import cli
from milc.questions import yesno
from qmk import submodules
from qmk.constants import QMK_FIRMWARE
from qmk.questions import yesno
from qmk.commands import run

183
lib/python/qmk/questions.py
View file

@ -1,183 +0,0 @@
"""Functions to collect user input.
"""
from milc import cli
try:
from milc import format_ansi
except ImportError:
from milc.ansi import format_ansi
def yesno(prompt, *args, default=None, **kwargs):
"""Displays prompt to the user and gets a yes or no response.
Returns True for a yes and False for a no.
If you add `--yes` and `--no` arguments to your program the user can answer questions by passing command line flags.
@add_argument('-y', '--yes', action='store_true', arg_only=True, help='Answer yes to all questions.')
@add_argument('-n', '--no', action='store_true', arg_only=True, help='Answer no to all questions.')
Arguments:
prompt
The prompt to present to the user. Can include ANSI and format strings like milc's `cli.echo()`.
default
Whether to default to a Yes or No when the user presses enter.
None- force the user to enter Y or N
True- Default to yes
False- Default to no
"""
if not args and kwargs:
args = kwargs
if 'no' in cli.args and cli.args.no:
return False
if 'yes' in cli.args and cli.args.yes:
return True
if default is not None:
if default:
prompt = prompt + ' [Y/n] '
else:
prompt = prompt + ' [y/N] '
while True:
cli.echo('')
answer = input(format_ansi(prompt % args))
cli.echo('')
if not answer and prompt is not None:
return default
elif answer.lower() in ['y', 'yes']:
return True
elif answer.lower() in ['n', 'no']:
return False
def question(prompt, *args, default=None, confirm=False, answer_type=str, validate=None, **kwargs):
"""Prompt the user to answer a question with a free-form input.
Arguments:
prompt
The prompt to present to the user. Can include ANSI and format strings like milc's `cli.echo()`.
default
The value to return when the user doesn't enter any value. Use None to prompt until they enter a value.
confirm
Present the user with a confirmation dialog before accepting their answer.
answer_type
Specify a type function for the answer. Will re-prompt the user if the function raises any errors. Common choices here include int, float, and decimal.Decimal.
validate
This is an optional function that can be used to validate the answer. It should return True or False and have the following signature:
def function_name(answer, *args, **kwargs):
"""
if not args and kwargs:
args = kwargs
if default is not None:
prompt = '%s [%s] ' % (prompt, default)
while True:
cli.echo('')
answer = input(format_ansi(prompt % args))
cli.echo('')
if answer:
if validate is not None and not validate(answer, *args, **kwargs):
continue
elif confirm:
if yesno('Is the answer "%s" correct?', answer, default=True):
try:
return answer_type(answer)
except Exception as e:
cli.log.error('Could not convert answer (%s) to type %s: %s', answer, answer_type.__name__, str(e))
else:
try:
return answer_type(answer)
except Exception as e:
cli.log.error('Could not convert answer (%s) to type %s: %s', answer, answer_type.__name__, str(e))
elif default is not None:
return default
def choice(heading, options, *args, default=None, confirm=False, prompt='Please enter your choice: ', **kwargs):
"""Present the user with a list of options and let them pick one.
Users can enter either the number or the text of their choice.
This will return the value of the item they choose, not the numerical index.
Arguments:
heading
The text to place above the list of options.
options
A sequence of items to choose from.
default
The index of the item to return when the user doesn't enter any value. Use None to prompt until they enter a value.
confirm
Present the user with a confirmation dialog before accepting their answer.
prompt
The prompt to present to the user. Can include ANSI and format strings like milc's `cli.echo()`.
"""
if not args and kwargs:
args = kwargs
if prompt and default:
prompt = prompt + ' [%s] ' % (default + 1,)
while True:
# Prompt for an answer.
cli.echo('')
cli.echo(heading % args)
cli.echo('')
for i, option in enumerate(options, 1):
cli.echo('\t{fg_cyan}%d.{fg_reset} %s', i, option)
cli.echo('')
answer = input(format_ansi(prompt))
cli.echo('')
# If the user types in one of the options exactly use that
if answer in options:
return answer
# Massage the answer into a valid integer
if answer == '' and default:
answer = default
else:
try:
answer = int(answer) - 1
except Exception:
# Normally we would log the exception here, but in the interest of clean UI we do not.
cli.log.error('Invalid choice: %s', answer + 1)
continue
# Validate the answer
if answer >= len(options) or answer < 0:
cli.log.error('Invalid choice: %s', answer + 1)
continue
if confirm and not yesno('Is the answer "%s" correct?', answer + 1, default=True):
continue
# Return the answer they chose.
return options[answer]