QueuedJobService
class QueuedJobService (View source)
A service that can be used for starting, stopping and listing queued jobs.
When a job is first added, it is initialised, its job type determined, then persisted to the database
When the queues are scanned, a job is reloaded and processed. Ignoring the persistence and reloading, it looks something like
job->getJobType(); job->getJobData(); data->write(); job->setup(); while !job->isComplete job->process(); job->getJobData(); data->write();
Traits
Provides extensions to this object to integrate it with standard config API methods.
A class that can be instantiated or replaced via DI
Allows an object to have extensions applied to it.
Allows an object to declare a set of custom methods
Config options
| extensions | array | An array of extension names and parameters to be applied to this object upon construction. |
from Extensible |
| unextendable_classes | array | Classes that cannot be extended |
from Extensible |
| stall_threshold | int | ||
| memory_limit | int | How much ram will we allow before pausing and releasing the memory? |
|
| time_limit | int | Optional time limit (in seconds) to run the service before restarting to release resources. |
|
| disable_health_check | bool | Disable health checks that usually occur when a runner first picks up a queue. Note that completely disabling health checks could result in many jobs that are always marked as running - that will never be restarted. If this option is disabled you may alternatively use the build task |
|
| max_init_jobs | int | Maximum number of jobs that can be initialised at any one time. |
|
| worker_ttl | string | Duration for TTL of queue workers based on ISO 8601 duration specification. |
|
| initialising_state_ttl | string | Duration for TTL of initialising state based on ISO 8601 duration specification. |
|
| use_shutdown_function | bool | Should "immediate" jobs be managed using the shutdown function? |
|
| cache_dir | string | The location for immediate jobs to be stored in |
|
| lock_file_enabled | bool | Maintenance lock file feature enabled / disable setting |
|
| lock_file_name | string | Maintenance lock file name |
|
| lock_file_path | string | Maintenance lock path (relative path starting at the base folder) Note that this path needs to point to a folder on a shared drive if multiple instances are used |
Properties
| protected static | array | $extra_methods | Custom method sources |
from CustomMethods |
| protected | array | $extra_method_registers | Name of methods to invoke by defineMethods for this instance |
from CustomMethods |
| protected static | array | $built_in_methods | Non-custom public methods. |
from CustomMethods |
| protected | Extension[] | $extension_instances | from Extensible | |
| protected | callable[][] | $beforeExtendCallbacks | List of callbacks to call prior to extensions having extend called on them, each grouped by methodName. |
from Extensible |
| protected | callable[][] | $afterExtendCallbacks | List of callbacks to call after extensions having extend called on them, each grouped by methodName. |
from Extensible |
| protected | int | $startedAt | Timestamp (in seconds) when the queue was started |
|
| public | DefaultQueueHandler | $queueHandler | ||
| public | TaskRunnerEngine | $queueRunner | ||
| public | array | $defaultJobs | Config controlled list of default/required jobs |
Methods
Get a configuration accessor for this class. Short hand for Config::inst()->get($this->class, .....).
Gets the uninherited value for the given config option
An implementation of the factory method, allows you to create an instance of a class
Creates a class instance by the "singleton" design pattern.
Attempts to locate and call a method dynamically added to a class at runtime if a default cannot be located
Adds any methods from Extension instances attached to this object.
Register an callback to invoke that defines extra methods
Return TRUE if a method exists on this object
Determines if a custom method with this name is defined.
Get meta-data details on a named method
Return the names of all the methods available on this object
Get all public built in methods for this class
Find all methods on the given object.
Add all the methods from an object property.
Add all the methods from an object property (which is an Extension) to this object.
Add a wrapper method - a method which points to another method with a different name. For example, Thumbnail(x) can be wrapped to generateThumbnail(x)
Add callback as a method.
Allows user code to hook into Object::extend prior to control being delegated to extensions. Each callback will be reset once called.
Allows user code to hook into Object::extend after control being delegated to extensions. Each callback will be reset once called.
Adds any methods from Extension instances attached to this object.
Add an extension to a specific class.
No description
Get extra config sources for this class
Return TRUE if a class has a specified extension.
Calls a method if available on both this object and all applied Extensions, and then attempts to merge all results into an array
Run the given function on all of this object's extensions. Note that this method originally returned void, so if you wanted to return results, you're hosed
Get an extension instance attached to this object by name.
Returns TRUE if this object instance has a specific extension applied in $extension_instances. Extension instances are initialized at constructor time, meaning if you use add_extension() afterwards, the added extension will just be added to new instances of the extended class. Use the static method has_extension() to check if a class (not an instance) has a specific extension.
Get all extension instances for this specific object instance.
Start a job (or however the queue handler determines it should be started)
Copies data from a job into a descriptor for persisting
Check the current job queues and see if any of the jobs currently in there should be started. If so, return the next job that should be executed
Runs an explicit check on all currently running jobs to make sure their "processed" count is incrementing between each run. If it's not, then we need to flag it as paused due to an error.
Checks through ll the scheduled jobs that are expected to exist
Prepares the given jobDescriptor for execution. Returns the job that will actually be run in a state ready for executing.
Given a QueuedJobDescriptor mark the job as initialised. Works sort of like a mutex.
Provides a wrapper when executing arbitrary code contained in job implementation this ensures that job specific code doesn't alter the configuration of the queue runner execution
No description
Determines the memory limit (in bytes) for this application Limits to the smaller of memory_limit configured via php.ini or silverstripe config
Gets a list of all the current jobs (or jobs that have recently finished)
Return the SQL filter used to get the job list - this is used by the UI for displaying the job list.
Get expiry time for a worker to be operating on a job, helps to identify jobs that have stalled more accurately.
Get expiry time for a INIT state of a queued job this helps to identify jobs that have stalled more accurately
Release job lock on the descriptor so it can run again
Mark a Job as Broken and release the lock so it can be resumed
Details
static Config_ForClass
config()
Get a configuration accessor for this class. Short hand for Config::inst()->get($this->class, .....).
mixed
uninherited(string $name)
Gets the uninherited value for the given config option
static Injectable
create(mixed ...$args)
An implementation of the factory method, allows you to create an instance of a class
This method will defer class substitution to the Injector API, which can be customised via the Config API to declare substitution classes.
This can be called in one of two ways - either calling via the class directly, or calling on Object and passing the class name as the first parameter. The following are equivalent: $list = DataList::create(SiteTree::class); $list = SiteTree::get();
static Injectable
singleton(string $class = null)
Creates a class instance by the "singleton" design pattern.
It will always return the same instance for this class, which can be used for performance reasons and as a simple way to access instance methods which don't rely on instance data (e.g. the custom SilverStripe static handling).
mixed
__call(string $method, array $arguments)
Attempts to locate and call a method dynamically added to a class at runtime if a default cannot be located
You can add extra methods to a class using Extensions}, {@link Object::createMethod() or Object::addWrapperMethod()
protected
defineMethods()
Adds any methods from Extension instances attached to this object.
All these methods can then be called directly on the instance (transparently mapped through __call()}), or called explicitly through {@link extend().
protected
registerExtraMethodCallback(string $name, callable $callback)
Register an callback to invoke that defines extra methods
bool
hasMethod(string $method)
Return TRUE if a method exists on this object
This should be used rather than PHP's inbuild method_exists() as it takes into account methods added via extensions
protected bool
hasCustomMethod($method)
Determines if a custom method with this name is defined.
protected array
getExtraMethodConfig(string $method)
Get meta-data details on a named method
array
allMethodNames(bool $custom = false)
Return the names of all the methods available on this object
static protected array
findBuiltInMethods(string|object $class = null)
Get all public built in methods for this class
protected array
findMethodsFrom(object $object)
Find all methods on the given object.
protected
addMethodsFrom(string $property, string|int $index = null)
Add all the methods from an object property.
protected
removeMethodsFrom(string $property, string|int $index = null)
Add all the methods from an object property (which is an Extension) to this object.
protected
addWrapperMethod(string $method, string $wrap)
Add a wrapper method - a method which points to another method with a different name. For example, Thumbnail(x) can be wrapped to generateThumbnail(x)
protected
addCallbackMethod(string $method, callable $callback)
Add callback as a method.
protected
beforeExtending(string $method, callable $callback)
Allows user code to hook into Object::extend prior to control being delegated to extensions. Each callback will be reset once called.
protected
afterExtending(string $method, callable $callback)
Allows user code to hook into Object::extend after control being delegated to extensions. Each callback will be reset once called.
protected
defineExtensionMethods()
Adds any methods from Extension instances attached to this object.
All these methods can then be called directly on the instance (transparently mapped through __call()}), or called explicitly through {@link extend().
static bool
add_extension(string $classOrExtension, string $extension = null)
Add an extension to a specific class.
The preferred method for adding extensions is through YAML config, since it avoids autoloading the class, and is easier to override in more specific configurations.
As an alternative, extensions can be added to a specific class directly in the Object::$extensions array. See SiteTree::$extensions for examples. Keep in mind that the extension will only be applied to new instances, not existing ones (including all instances created through singleton()).
static
remove_extension(string $extension)
Remove an extension from a class.
Note: This will not remove extensions from parent classes, and must be called directly on the class assigned the extension.
Keep in mind that this won't revert any datamodel additions of the extension at runtime, unless its used before the schema building kicks in (in your _config.php). Doesn't remove the extension from any Object instances which are already created, but will have an effect on new extensions. Clears any previously created singletons through singleton() to avoid side-effects from stale extension information.
static array
get_extensions(string $class = null, bool $includeArgumentString = false)
No description
static array|null
get_extra_config_sources(string $class = null)
Get extra config sources for this class
static bool
has_extension(string $classOrExtension, string $requiredExtension = null, bool $strict = false)
Return TRUE if a class has a specified extension.
This supports backwards-compatible format (static Object::has_extension($requiredExtension)) and new format ($object->has_extension($class, $requiredExtension))
array
invokeWithExtensions(string $method, mixed ...$arguments)
Calls a method if available on both this object and all applied Extensions, and then attempts to merge all results into an array
array
extend(string $method, mixed ...$arguments)
Run the given function on all of this object's extensions. Note that this method originally returned void, so if you wanted to return results, you're hosed
Currently returns an array, with an index resulting every time the function is called. Only adds returns if they're not NULL, to avoid bogus results from methods just defined on the parent extension. This is important for permission-checks through extend, as they use min() to determine if any of the returns is FALSE. As min() doesn't do type checking, an included NULL return would fail the permission checks.
The extension methods are defined during __construct()} in {@link defineMethods().
Extension|null
getExtensionInstance(string $extension)
Get an extension instance attached to this object by name.
bool
hasExtension(string $extension)
Returns TRUE if this object instance has a specific extension applied in $extension_instances. Extension instances are initialized at constructor time, meaning if you use add_extension() afterwards, the added extension will just be added to new instances of the extended class. Use the static method has_extension() to check if a class (not an instance) has a specific extension.
Caution: Don't use singleton(
Extension[]
getExtensionInstances()
Get all extension instances for this specific object instance.
See get_extensions() to get all applied extension classes for this class (not the instance).
This method also provides lazy-population of the extension_instances property.
__construct()
QueuedJobService constructor.
int
queueJob(QueuedJob $job, null|string $startAfter = null, null|int $userId = null, null|int $queueName = null)
Adds a job to the queue to be started
startJob(QueuedJobDescriptor $jobDescriptor, string $startAfter = null)
Start a job (or however the queue handler determines it should be started)
bool
isAtMaxJobs()
Check if maximum number of jobs are currently initialised.
protected
copyJobToDescriptor(QueuedJob $job, QueuedJobDescriptor $jobDescriptor)
Copies data from a job into a descriptor for persisting
protected
copyDescriptorToJob(QueuedJobDescriptor $jobDescriptor, QueuedJob $job)
No description
QueuedJobDescriptor|null
getNextPendingJob(string $type = null)
Check the current job queues and see if any of the jobs currently in there should be started. If so, return the next job that should be executed
array
checkJobHealth(int $queue = null)
Runs an explicit check on all currently running jobs to make sure their "processed" count is incrementing between each run. If it's not, then we need to flag it as paused due to an error.
This typically happens when a PHP fatal error is thrown, which can't be picked up by the error handler or exception checker; in this case, we detect these stalled jobs later and fix (try) to fix them
This function returns the IDs of stalled and broken jobs this information can be used to implement some custom follow up actions such as sending email reports
checkDefaultJobs($queue = null)
Checks through ll the scheduled jobs that are expected to exist
protected
restartStalledJob(QueuedJobDescriptor $stalledJob)
Attempt to restart a stalled job
protected QueuedJob|bool
initialiseJob(QueuedJobDescriptor $jobDescriptor)
Prepares the given jobDescriptor for execution. Returns the job that will actually be run in a state ready for executing.
Note that this is called each time a job is picked up to be executed from the cron job - meaning that jobs that are paused and restarted will have 'setup()' called on them again, so your job MUST detect that and act accordingly.
protected bool
grabMutex(QueuedJobDescriptor $jobDescriptor)
Given a QueuedJobDescriptor mark the job as initialised. Works sort of like a mutex.
Currently a database lock isn't entirely achievable, due to database adapters not supporting locks. This may still have a race condition, but this should minimise the possibility. Side effect is the job status will be changed to "Initialised".
Assumption is the job has a status of "Queued" or "Wait".
bool
runJob(int $jobId)
Start the actual execution of a job.
The assumption is the jobID refers to a QueuedJobDescriptor that is status set as "Queued".
This method will continue executing until the job says it's completed
protected
finaliseLogging(LoggerInterface $logger)
No description
protected mixed
withNestedState(callable $callback)
Provides a wrapper when executing arbitrary code contained in job implementation this ensures that job specific code doesn't alter the configuration of the queue runner execution
protected
handleJobInitialisationException(QueuedJobDescriptor $jobDescriptor, Throwable $e)
No description
protected
handleBrokenJobException(QueuedJobDescriptor $jobDescriptor, QueuedJob $job, Exception|Throwable $e)
No description
protected
markStarted()
Start timer
protected bool
hasPassedTimeLimit()
Is execution time too long?
protected bool
isMemoryTooHigh()
Is memory usage too high?
protected float
getMemoryUsage()
Get peak memory usage of this application
protected float
getMemoryLimit()
Determines the memory limit (in bytes) for this application Limits to the smaller of memory_limit configured via php.ini or silverstripe config
protected float
getPHPMemoryLimit()
Calculate the current memory limit of the server
protected float
parseMemory(string $memString)
Convert memory limit string to bytes.
Based on implementation in install.php5
protected
humanReadable($size)
No description
QueuedJobDescriptor>
getJobList(string $type = null, int $includeUpUntil = 0)
Gets a list of all the current jobs (or jobs that have recently finished)
string
getJobListFilter(string $type = null, int $includeUpUntil = 0)
Return the SQL filter used to get the job list - this is used by the UI for displaying the job list.
..
runQueue(string $queue)
Process the job queue with the current queue runner
processJobQueue(string $name)
Process all jobs from a given queue
onShutdown()
When PHP shuts down, we want to process all of the immediate queue items
We use the 'getNextPendingJob' method, instead of just iterating the queue, to ensure we ignore paused or stalled jobs.
LoggerInterface
getLogger()
Get a logger
setLogger(LoggerInterface $logger)
No description
enableMaintenanceLock()
No description
disableMaintenanceLock()
No description
bool
isMaintenanceLockActive()
No description
protected string
getWorkerExpiry()
Get expiry time for a worker to be operating on a job, helps to identify jobs that have stalled more accurately.
protected string
getInitStateExpiry()
Get expiry time for a INIT state of a queued job this helps to identify jobs that have stalled more accurately
protected void
releaseJobLock(QueuedJobDescriptor $descriptor)
Release job lock on the descriptor so it can run again
protected void
markJobAsBroken(QueuedJobDescriptor $descriptor)
Mark a Job as Broken and release the lock so it can be resumed