endian.job.generic package

Submodules

endian.job.generic.apachejob module

Module author: Davide Alberani

class ApacheJob

Bases: endian.job.job.Job

Base class for jobs that need to manage the apache configuration.

conf = None

mods = None

sites = None

update_apache_links(update=None)

Update symlink in Apache’s configuration directories.

write_httpd_ports(section, ports)

Write a /var/efw/{section}/httpd_ports csv file with the format ip_address,port

endian.job.generic.crawlerjob module

Module author: Alber Maximilian

class CrawlerJob

Bases: endian.job.job.Job

A job to crawl in the file system and perform actions on the files.

You should overwrite:

• crawl

You may set:

• settings_file_name
• pattern
• pattern_key

You may overwrite:

• get_settings_file_name
• get_pattern
• pre
• post

crawl(file)

Should be overwritten.

Hook to perform a operation on the crawled file.

Parameters:file (Strings) – The filename. Returns:The filename of the, may changed file or None. Return type:String

get_pattern()

Should be overwritten.

Returns the search pattern.

In the default implemenation the function looks, if a the settings_file_name and a pattern_key is defined and reads out of the settings file the pattern. If not it uses the member pattern. If even this not exists an empty string will be returned.

Returns:The search patterns. Return type:list of strings String or String

get_settings_file_name()

May returns the settings file used by this job, otherwise an empty string.

As default implementation this function return the class member settings_file_name.

Returns:May the settings file used by this job or an empty string. Return type:String

load_config()

Loads the settings file named by the function get_settings_file_name into the member config_values.

post(files)

Should be overwritten.

Post hook, is triggered after the crawling.

Parameters:files (list of Strings) – The list of found files. Returns:The return value of the start function. Return type:Not specified

pre(files)

Should be overwritten.

Prepartion hook, is triggered before the crawling.

Parameters:files (list of Strings) – The list of found files. Returns:The list of found files. Return type:list of Strings

restart(job, *args, **argv)

Starts the job. :param options: A dict with options. :type options: dict

start(job, *args, **argv)

Starts the job. :param options: A dict with options. :type options: dict

start_crawling(options)

Takes the pattern to search for files. The founded filenames are filtered by pre. Then for each filename crawl will performed. In the end post will called with the filenames.

Parameters:options (dict) – A dict with options.

endian.job.generic.cronjob module

Module author: Alber Maximilian

class CronJob

Bases: endian.job.generic.cronjob.GenericCronJob

A job to schedule cron scripts.

Following functions can/should be overwritten:

• get_scripts

Following function may be overwritten:

• get_settings_file_name

class AnaCronJob

Bases: endian.job.generic.cronjob.GenericCronJob

A job to schedule anacron scripts.

Following functions can/should be overwritten:

• get_scripts

Following function may be overwritten:

• get_settings_file_name

endian.job.generic.downloadjob module

Module author: Alber Maximilian

class DownLoadJob

Bases: endian.job.job.Job

A generic job to schedule file downloads and trigger pre and post actions.

You can or overwrite the hook function or use their default implementations and set members.

Following functions can/should be overwritten: * should_update * get_urls * pre * post_un_compress * post

Following function may be overwritten:

• get_http_authentication
• get_settings_file_name

You may set one of this members:

• settings_file_name
• urls, like described in get_urls

create_tmp_dir()

Returns the tmp directory to use.

Returns:The directory name. Return type:String

delete_tmp_dir(tmp_dir)

If the directory was created by the job, the function will delete it.

Parameters:tmp_dir (String) – The tmp directory.

download(tmp_dir, urls)

Downloads the urls and stores them in the tmp directory. Returns the urls parameter with a mapping between urls and tmp files(key tmp_file). If a settings file is specified, the etag_key is set in the urls dict and the url has a etag, it will be saved.

Parameters:

• tmp_dir (String) – The tmp directory.
• urls (list of strings) – The urls to download.

Returns:

The urls parameter, each dict extened with the key tmp_file

Return type:

list of dicts

get_http_authentication()

May be overwritten.

Returns a tuple with username and password for a eventually http authentication. If no authentication is necessary it will return None.

The default implementation will look, if in the options is httpusername and httppassword setted, the the those are returned, otherwise the function will lookup self.config_values for a credentials entry. If this is set to ‘off’ None will returned, otherwise the functions will look in ‘{entry}/settings’ for ‘httpusername’ and ‘httppassword’ and return those.

Returns:A username and password tuple or None. Return type:A string tuple or None

get_settings_file_name()

May returns the settings file used by this job, otherwise an empty string.

As default implementation this function return the class member settings_file_name.

Returns:May the settings file used by this job or an empty string. Return type:String

get_urls()

Should be overwritten.

Returns dicts with information about the urls to download.

The dict will have following keys:

• url: the url.

The dict will mey have following keys:

• etag: the etag of the last download.

• uncompress: True, if the url returns a tar.gz

  file which should be uncompressed.

• location: The location, where the download is

  to save.

• archive_prefix: The prefix-“directory” in the

  archive.

In the default implementation the function will use, if exists, the member urls to create the dicts. If url_key, etag_key, uncompress_key, location_key or archive_prefix_key is given in the dict, the function will lookup in the settings file for the value. If the key is not given or the value is empty, the value referenced by the key url, etag, uncompress, location or archive_prefix will used. If both keys are missing no entry will made in the return dict. For the etag you can also a specify a different settings file using the key etag_file.

Returns:A list of url dicts. Return type:list of dicts

load_config()

Loads the settings file named by the function get_settings_file_name into the member config_values.

post(tmp_dir, urls)

Should be overwritten.

Post hook, is triggered after the download and the possible uncompressing.

In the default implementation the content of urls with a location key in the urls parameter will be moved to this location.

Parameters:

• tmp_dir (String) – The temporary directory for the downloads.
• urls (list of dicts) – The urls to downloaded.

post_un_compress(tmp_dir, urls)

Should be overwritten.

Post hook to eventually uncompress the files, is triggered after the download and before the post hoook.

In the default implementation the urls with uncompress as key in the urls parameter will be uncompressed and the key download_location will be added to the urls dict.

Parameters:

• tmp_dir (String) – The temporary directory for the downloads.
• urls (list of dicts) – The urls to downloaded.

Returns:

The url list to pass to the post hook.

Return type:

list of dicts

pre(tmp_dir, urls)

Should be overwritten.

Prepartion hook, is triggered before the download.

Parameters:

• tmp_dir (String) – The temporary directory for the downloads.
• urls (list of dicts) – The urls to download.

Returns:

The url list to pass to the download.

Return type:

list of dicts

restart(job, *args, **argv)

Starts the job.

Parameters:options (dict) – A dict with options.

set_timestamps(url_defs)

Set the timestamp after a successful download.

should_update(urls)

Should be overwritten.

Returns True, if there is new content to download.

In the default implementation the function will return true, if no etag key is in the url dict otherwise will controll, if the etag has changed.

Parameters:urls (list of dicts) – The urls to download. Returns:The urls to update. Return type:list of dicts

start(options)

Starts the download jobs. In detail controlls, if there is new data. If so, launches the pre hook, downloads the urls retrieved by getURLS, lanaunches the postUnCompress and then the post hook.

Parameters:options (dict) – A dict with options.

endian.job.generic.handlersjob module

class HandlersJob

Bases: endian.job.job.Job

expand_vars(text, environ)

Expand environ variables of form $var and ${var}. Unknown variables are left unchanged.

is_run_parts_valid_name(filename)

Check if the filename is a valid filename for run-parts Valid filenames should consist entirely of upper and lower case letters, digits, underscores (‘_’), and hyphens (‘-‘).

Parameters:filename (string) – filename Returns:True if the filename is a run-parts valid filename, False otherwise Return type:bool

parse_jobsengine_filename(filepath, filename=None, environ=None)

Parse a jobsengine filename (parse the filename, not the content), in the format: [PRIORITY].cmd.args where PRIORITY is a number, cmd is a cmd in ‘start’, ‘stop’, ‘restart’, ‘emit’, ‘call’, and ‘request’ args are the mandatory cmd arguments

If the file is not empty, the file content is used as additional argument. Env variables of form $var and ${var} in the file content are expanded, e.g. options(home:”${HOME}”) –> options(home:”/Users/andreax”)

Example of valid filename:

05.restart.xyz 05restart.xyz # missing ‘.’ after priority, but accepted 05.restart.sync.xyz restart.xyz restart.sync.xyz

Parameters:

• filepath (string) – absolute file path
• filename (string) – filename (without path) - If empty, extract the filename from path

Returns:

a cmd to send to the jobsengine (e.g. ‘start ssh’) or None

Return type:

string

run_parts(directory, prefix=None, environ=None)

Runs a number of scripts and jobsengine commands found in a single directory directory. Scripts and jobsengine commands are recognized by the filename.

• scripts – Filenames should consist entirely of upper and lower case letters,

  digits, underscores, and hyphens (e.g. 01interfacemark)

• jobsengine cmd – Filename format [PRIORITY].cmd.args (e.g. 05.restart.xyz, restart.xyz)

  If the file is not empty, the file content is used as additional argument.

Parameters:

• directory (string) – directory path
• prefix (string) – prefix added to the jobsengine cmds
• environ (a dict or None) – if it is not None, it must be a mapping that defines the environment variables for the new process; these are used instead of inheriting the jobsengine environment, which is the default behavior.

endian.job.generic.servicejob module

Module author: Peter Warasin

class ServiceJob

Bases: endian.job.job.Job

actions = None

call_action(service, options, serviceChanged=True)

configFiles = []

config_values = None

dependingJobs = None

enabledkeys = []

execute_actions(options, changed={})

execute_dependencies(options, action, hook)

hasService = True

load_config()

load_config_hook = None

restart(job, *args, **argv)

restart_hook = None

save_config(options)

save_hook = None

serviceBlocking = False

serviceCanReload = False

serviceName = None

settingsName = None

shouldstart(job, *args, **argv)

shouldstart_hook = None

start(job, *args, **argv)

start_hook = None

stop(job, *args, **argv)

stop_hook = None

updatewizard(job, *args, **argv)