mirror of
https://github.com/deepfakes/faceswap
synced 2025-06-07 10:43:27 -04:00
* model_refactor (#571) * original model to new structure * IAE model to new structure * OriginalHiRes to new structure * Fix trainer for different resolutions * Initial config implementation * Configparse library added * improved training data loader * dfaker model working * Add logging to training functions * Non blocking input for cli training * Add error handling to threads. Add non-mp queues to queue_handler * Improved Model Building and NNMeta * refactor lib/models * training refactor. DFL H128 model Implementation * Dfaker - use hashes * Move timelapse. Remove perceptual loss arg * Update INSTALL.md. Add logger formatting. Update Dfaker training * DFL h128 partially ported * Add mask to dfaker (#573) * Remove old models. Add mask to dfaker * dfl mask. Make masks selectable in config (#575) * DFL H128 Mask. Mask type selectable in config. * remove gan_v2_2 * Creating Input Size config for models Creating Input Size config for models Will be used downstream in converters. Also name change of image_shape to input_shape to clarify ( for future models with potentially different output_shapes) * Add mask loss options to config * MTCNN options to config.ini. Remove GAN config. Update USAGE.md * Add sliders for numerical values in GUI * Add config plugins menu to gui. Validate config * Only backup model if loss has dropped. Get training working again * bugfixes * Standardise loss printing * GUI idle cpu fixes. Graph loss fix. * mutli-gpu logging bugfix * Merge branch 'staging' into train_refactor * backup state file * Crash protection: Only backup if both total losses have dropped * Port OriginalHiRes_RC4 to train_refactor (OriginalHiRes) * Load and save model structure with weights * Slight code update * Improve config loader. Add subpixel opt to all models. Config to state * Show samples... wrong input * Remove AE topology. Add input/output shapes to State * Port original_villain (birb/VillainGuy) model to faceswap * Add plugin info to GUI config pages * Load input shape from state. IAE Config options. * Fix transform_kwargs. Coverage to ratio. Bugfix mask detection * Suppress keras userwarnings. Automate zoom. Coverage_ratio to model def. * Consolidation of converters & refactor (#574) * Consolidation of converters & refactor Initial Upload of alpha Items - consolidate convert_mased & convert_adjust into one converter -add average color adjust to convert_masked -allow mask transition blur size to be a fixed integer of pixels and a fraction of the facial mask size -allow erosion/dilation size to be a fixed integer of pixels and a fraction of the facial mask size -eliminate redundant type conversions to avoid multiple round-off errors -refactor loops for vectorization/speed -reorganize for clarity & style changes TODO - bug/issues with warping the new face onto a transparent old image...use a cleanup mask for now - issues with mask border giving black ring at zero erosion .. investigate - remove GAN ?? - test enlargment factors of umeyama standard face .. match to coverage factor - make enlargment factor a model parameter - remove convert_adjusted and referencing code when finished * Update Convert_Masked.py default blur size of 2 to match original... description of enlargement tests breakout matrxi scaling into def * Enlargment scale as a cli parameter * Update cli.py * dynamic interpolation algorithm Compute x & y scale factors from the affine matrix on the fly by QR decomp. Choose interpolation alogrithm for the affine warp based on an upsample or downsample for each image * input size input size from config * fix issues with <1.0 erosion * Update convert.py * Update Convert_Adjust.py more work on the way to merginf * Clean up help note on sharpen * cleanup seamless * Delete Convert_Adjust.py * Update umeyama.py * Update training_data.py * swapping * segmentation stub * changes to convert.str * Update masked.py * Backwards compatibility fix for models Get converter running * Convert: Move masks to class. bugfix blur_size some linting * mask fix * convert fixes - missing facehull_rect re-added - coverage to % - corrected coverage logic - cleanup of gui option ordering * Update cli.py * default for blur * Update masked.py * added preliminary low_mem version of OriginalHighRes model plugin * Code cleanup, minor fixes * Update masked.py * Update masked.py * Add dfl mask to convert * histogram fix & seamless location * update * revert * bugfix: Load actual configuration in gui * Standardize nn_blocks * Update cli.py * Minor code amends * Fix Original HiRes model * Add masks to preview output for mask trainers refactor trainer.__base.py * Masked trainers converter support * convert bugfix * Bugfix: Converter for masked (dfl/dfaker) trainers * Additional Losses (#592) * initial upload * Delete blur.py * default initializer = He instead of Glorot (#588) * Allow kernel_initializer to be overridable * Add ICNR Initializer option for upscale on all models. * Hopefully fixes RSoDs with original-highres model plugin * remove debug line * Original-HighRes model plugin Red Screen of Death fix, take #2 * Move global options to _base. Rename Villain model * clipnorm and res block biases * scale the end of res block * res block * dfaker pre-activation res * OHRES pre-activation * villain pre-activation * tabs/space in nn_blocks * fix for histogram with mask all set to zero * fix to prevent two networks with same name * GUI: Wider tooltips. Improve TQDM capture * Fix regex bug * Convert padding=48 to ratio of image size * Add size option to alignments tool extract * Pass through training image size to convert from model * Convert: Pull training coverage from model * convert: coverage, blur and erode to percent * simplify matrix scaling * ordering of sliders in train * Add matrix scaling to utils. Use interpolation in lib.aligner transform * masked.py Import get_matrix_scaling from utils * fix circular import * Update masked.py * quick fix for matrix scaling * testing thus for now * tqdm regex capture bugfix * Minor ammends * blur size cleanup * Remove coverage option from convert (Now cascades from model) * Implement convert for all model types * Add mask option and coverage option to all existing models * bugfix for model loading on convert * debug print removal * Bugfix for masks in dfl_h128 and iae * Update preview display. Add preview scaling to cli * mask notes * Delete training_data_v2.py errant file * training data variables * Fix timelapse function * Add new config items to state file for legacy purposes * Slight GUI tweak * Raise exception if problem with loaded model * Add Tensorboard support (Logs stored in model directory) * ICNR fix * loss bugfix * convert bugfix * Move ini files to config folder. Make TensorBoard optional * Fix training data for unbalanced inputs/outputs * Fix config "none" test * Keep helptext in .ini files when saving config from GUI * Remove frame_dims from alignments * Add no-flip and warp-to-landmarks cli options * Revert OHR to RC4_fix version * Fix lowmem mode on OHR model * padding to variable * Save models in parallel threads * Speed-up of res_block stability * Automated Reflection Padding * Reflect Padding as a training option Includes auto-calculation of proper padding shapes, input_shapes, output_shapes Flag included in config now * rest of reflect padding * Move TB logging to cli. Session info to state file * Add session iterations to state file * Add recent files to menu. GUI code tidy up * [GUI] Fix recent file list update issue * Add correct loss names to TensorBoard logs * Update live graph to use TensorBoard and remove animation * Fix analysis tab. GUI optimizations * Analysis Graph popup to Tensorboard Logs * [GUI] Bug fix for graphing for models with hypens in name * [GUI] Correctly split loss to tabs during training * [GUI] Add loss type selection to analysis graph * Fix store command name in recent files. Switch to correct tab on open * [GUI] Disable training graph when 'no-logs' is selected * Fix graphing race condition * rename original_hires model to unbalanced
240 lines
9.8 KiB
Python
240 lines
9.8 KiB
Python
#!/usr/bin python3
|
|
""" Cli Options for the GUI """
|
|
import inspect
|
|
from argparse import SUPPRESS
|
|
import logging
|
|
from tkinter import ttk
|
|
|
|
from lib import cli
|
|
import tools.cli as ToolsCli
|
|
from .utils import get_images
|
|
|
|
logger = logging.getLogger(__name__) # pylint: disable=invalid-name
|
|
|
|
|
|
class CliOptions():
|
|
""" Class and methods for the command line options """
|
|
def __init__(self):
|
|
logger.debug("Initializing %s", self.__class__.__name__)
|
|
self.categories = ("faceswap", "tools")
|
|
self.commands = dict()
|
|
self.opts = dict()
|
|
self.build_options()
|
|
logger.debug("Initialized %s", self.__class__.__name__)
|
|
|
|
def build_options(self):
|
|
""" Get the commands that belong to each category """
|
|
for category in self.categories:
|
|
logger.debug("Building '%s'", category)
|
|
src = ToolsCli if category == "tools" else cli
|
|
mod_classes = self.get_cli_classes(src)
|
|
self.commands[category] = self.sort_commands(category, mod_classes)
|
|
self.opts.update(self.extract_options(src, mod_classes))
|
|
logger.debug("Built '%s'", category)
|
|
|
|
@staticmethod
|
|
def get_cli_classes(cli_source):
|
|
""" Parse the cli scripts for the arg classes """
|
|
mod_classes = list()
|
|
for name, obj in inspect.getmembers(cli_source):
|
|
if inspect.isclass(obj) and name.lower().endswith("args") \
|
|
and name.lower() not in (("faceswapargs",
|
|
"extractconvertargs",
|
|
"guiargs")):
|
|
mod_classes.append(name)
|
|
logger.debug(mod_classes)
|
|
return mod_classes
|
|
|
|
def sort_commands(self, category, classes):
|
|
""" Format classes into command names and sort:
|
|
Specific workflow order for faceswap.
|
|
Alphabetical for all others """
|
|
commands = sorted(self.format_command_name(command)
|
|
for command in classes)
|
|
if category == "faceswap":
|
|
ordered = ["extract", "train", "convert"]
|
|
commands = ordered + [command for command in commands
|
|
if command not in ordered]
|
|
logger.debug(commands)
|
|
return commands
|
|
|
|
@staticmethod
|
|
def format_command_name(classname):
|
|
""" Format args class name to command """
|
|
return classname.lower()[:-4]
|
|
|
|
def extract_options(self, cli_source, mod_classes):
|
|
""" Extract the existing ArgParse Options
|
|
into master options Dictionary """
|
|
subopts = dict()
|
|
for classname in mod_classes:
|
|
logger.debug("Processing: (classname: '%s')", classname)
|
|
command = self.format_command_name(classname)
|
|
options = self.get_cli_arguments(cli_source, classname, command)
|
|
options = self.process_options(options)
|
|
logger.debug("Processed: (classname: '%s', command: '%s', options: %s)",
|
|
classname, command, options)
|
|
subopts[command] = options
|
|
return subopts
|
|
|
|
@staticmethod
|
|
def get_cli_arguments(cli_source, classname, command):
|
|
""" Extract the options from the main and tools cli files """
|
|
meth = getattr(cli_source, classname)(None, command)
|
|
return meth.argument_list + meth.optional_arguments + meth.global_arguments
|
|
|
|
def process_options(self, command_options):
|
|
""" Process the options for a single command """
|
|
final_options = list()
|
|
for opt in command_options:
|
|
logger.trace("Processing: %s", opt)
|
|
if opt.get("help", "") == SUPPRESS:
|
|
logger.trace("Skipping suppressed option: %s", opt)
|
|
continue
|
|
ctl, sysbrowser, filetypes, action_option = self.set_control(opt)
|
|
opt["control_title"] = self.set_control_title(opt.get("opts", ""))
|
|
opt["control"] = ctl
|
|
opt["filesystem_browser"] = sysbrowser
|
|
opt["filetypes"] = filetypes
|
|
opt["action_option"] = action_option
|
|
final_options.append(opt)
|
|
logger.trace("Processed: %s", opt)
|
|
return final_options
|
|
|
|
@staticmethod
|
|
def set_control_title(opts):
|
|
""" Take the option switch and format it nicely """
|
|
ctltitle = opts[1] if len(opts) == 2 else opts[0]
|
|
ctltitle = ctltitle.replace("-", " ").replace("_", " ").strip().title()
|
|
return ctltitle
|
|
|
|
def set_control(self, option):
|
|
""" Set the control and filesystem browser to use for each option """
|
|
sysbrowser = None
|
|
action = option.get("action", None)
|
|
action_option = option.get("action_option", None)
|
|
filetypes = option.get("filetypes", None)
|
|
ctl = ttk.Entry
|
|
if action in (cli.FullPaths,
|
|
cli.DirFullPaths,
|
|
cli.FileFullPaths,
|
|
cli.DirOrFileFullPaths,
|
|
cli.SaveFileFullPaths,
|
|
cli.ContextFullPaths):
|
|
sysbrowser, filetypes = self.set_sysbrowser(action,
|
|
filetypes,
|
|
action_option)
|
|
elif option.get("min_max", None):
|
|
ctl = ttk.Scale
|
|
elif option.get("choices", "") != "":
|
|
ctl = ttk.Combobox
|
|
elif option.get("action", "") == "store_true":
|
|
ctl = ttk.Checkbutton
|
|
return ctl, sysbrowser, filetypes, action_option
|
|
|
|
@staticmethod
|
|
def set_sysbrowser(action, filetypes, action_option):
|
|
""" Set the correct file system browser and filetypes
|
|
for the passed in action """
|
|
sysbrowser = ["folder"]
|
|
filetypes = "default" if not filetypes else filetypes
|
|
if action == cli.FileFullPaths:
|
|
sysbrowser = ["load"]
|
|
elif action == cli.SaveFileFullPaths:
|
|
sysbrowser = ["save"]
|
|
elif action == cli.DirOrFileFullPaths:
|
|
sysbrowser = ["folder", "load"]
|
|
elif action == cli.ContextFullPaths and action_option:
|
|
sysbrowser = ["context"]
|
|
logger.debug("sysbrowser: %s, filetypes: '%s'", sysbrowser, filetypes)
|
|
return sysbrowser, filetypes
|
|
|
|
def set_context_option(self, command):
|
|
""" Set the tk_var for the source action option
|
|
that dictates the context sensitive file browser. """
|
|
actions = {item["opts"][0]: item["value"]
|
|
for item in self.gen_command_options(command)}
|
|
for opt in self.gen_command_options(command):
|
|
if opt["filesystem_browser"] == ["context"]:
|
|
opt["action_option"] = actions[opt["action_option"]]
|
|
|
|
def gen_command_options(self, command):
|
|
""" Yield each option for specified command """
|
|
for option in self.opts[command]:
|
|
yield option
|
|
|
|
def options_to_process(self, command=None):
|
|
""" Return a consistent object for processing
|
|
regardless of whether processing all commands
|
|
or just one command for reset and clear """
|
|
if command is None:
|
|
options = [opt for opts in self.opts.values() for opt in opts]
|
|
else:
|
|
options = [opt for opt in self.gen_command_options(command)]
|
|
return options
|
|
|
|
def reset(self, command=None):
|
|
""" Reset the options for all or passed command
|
|
back to default value """
|
|
logger.debug("Resetting options to default. (command: '%s'", command)
|
|
for option in self.options_to_process(command):
|
|
default = option.get("default", "")
|
|
default = "" if default is None else default
|
|
if (option.get("nargs", None)
|
|
and isinstance(default, (list, tuple))):
|
|
default = ' '.join(str(val) for val in default)
|
|
option["value"].set(default)
|
|
|
|
def clear(self, command=None):
|
|
""" Clear the options values for all or passed
|
|
commands """
|
|
logger.debug("Clearing options. (command: '%s'", command)
|
|
for option in self.options_to_process(command):
|
|
if isinstance(option["value"].get(), bool):
|
|
option["value"].set(False)
|
|
elif isinstance(option["value"].get(), int):
|
|
option["value"].set(0)
|
|
else:
|
|
option["value"].set("")
|
|
|
|
def get_option_values(self, command=None):
|
|
""" Return all or single command control titles
|
|
with the associated tk_var value """
|
|
ctl_dict = dict()
|
|
for cmd, opts in self.opts.items():
|
|
if command and command != cmd:
|
|
continue
|
|
cmd_dict = dict()
|
|
for opt in opts:
|
|
cmd_dict[opt["control_title"]] = opt["value"].get()
|
|
ctl_dict[cmd] = cmd_dict
|
|
logger.debug("command: '%s', ctl_dict: '%s'", command, ctl_dict)
|
|
return ctl_dict
|
|
|
|
def get_one_option_variable(self, command, title):
|
|
""" Return a single tk_var for the specified
|
|
command and control_title """
|
|
for option in self.gen_command_options(command):
|
|
if option["control_title"] == title:
|
|
return option["value"]
|
|
return None
|
|
|
|
def gen_cli_arguments(self, command):
|
|
""" Return the generated cli arguments for
|
|
the selected command """
|
|
for option in self.gen_command_options(command):
|
|
optval = str(option.get("value", "").get())
|
|
opt = option["opts"][0]
|
|
if command in ("extract", "convert") and opt == "-o":
|
|
get_images().pathoutput = optval
|
|
if optval in ("False", ""):
|
|
continue
|
|
elif optval == "True":
|
|
yield (opt, )
|
|
else:
|
|
if option.get("nargs", None):
|
|
optval = optval.split(" ")
|
|
opt = [opt] + optval
|
|
else:
|
|
opt = (opt, optval)
|
|
yield opt
|