Functional equation modeling of adaptive operant-control systems via Matkowski fixed point theory

TL;DR

This paper introduces a new mathematical approach to model operant-control behavior without initial conditions, using fixed point theory.

Contribution

A generalized functional equation for operant-control systems is proposed, supported by Matkowski fixed point theory.

Findings

The generalized equation has a unique probabilistic solution, proven via Matkowski fixed point theorem.

Simulations confirm the validity of the theoretical framework for behavioral modeling.

Fixed point theory is shown to be effective in analyzing control-based behavioral models.

Abstract

This paper presents a generalized form of the functional equation used in operant-control models by removing the requirement for initial conditions. The proposed formulation extends earlier studies in mathematical psychology and provides a broader analytical framework for modeling operant-control behavior. Using the Matkowski fixed point theorem, we prove the existence and uniqueness of a probabilistic solution to the generalized equation. Illustrative examples and simulations are included to demonstrate the validity of the theoretical results. This work shows that fixed point theory can effectively support the formulation and analysis of control-based behavioral models.

Figures

Pigeon project chamber.

Design of fMRI experiment.

Sequential evolution of the iterative process.Illustration of the Picard iterative scheme applied to the operator T. (A): Initial approximation Γ0. (B): Picard iterate Γ1=TΓ0. (C): Picard iterate Γ2=TΓ1. (D): Picard iterate Γ3=TΓ2.

Sequential progression of Picard iterations demonstrating convergence toward the unique fixed point of T.(A): Initial approximation Γ0. (B): Picard iterate Γ1=TΓ0. (C): Picard iterate Γ2=TΓ1.

Equations

Full text

Introduction

Operant conditioning was first described by psychologist B.F. Skinner [1]. His theory rests on two fundamental assumptions: first, human and animal behavior is influenced by environmental factors; second, the consequences of a behavior determine its likelihood of being repeated. In fact, Behaviors that result in positive consequences are more likely to be repeated, while those that result in negative consequences are less likely to occur again [2–5].

Here, we present two experiments that inspired our investigation in this study. During World War II, B.F. Skinner [6] conducted research on Project Pigeon, also known as Project Orcon, an experimental initiative aimed at developing pigeon guided missiles. Skinner trained pigeons to peck at a target, rewarding them with food upon successful completion of the task. On the other hand, Gregory S. Berns et.al [7] conducted experiments involving water and juice, revealing a notable divergence in brain responses between prediction and subjective preferences [8]. Brain responses to preferences were predominantly cortical, whereas responses to predictability involved activation of reward systems targeted by dopaminergic neurons [9] and [10]. Further, several unresolved issues in this field merit further exploration.

In the separate line of research, the dynamic relationship between predators and their prey is a critical focus in ecosystem studies [11]. Recent research indicates that predation can significantly influence prey population size, acting as a top-down control mechanism. Indeed, observing the interplay between these population dynamics offers insights into long-term changes in population sizes over time [12].

In 1973, Lyubich and Shapiro [13] studied the existence and uniqueness of a continuous solution $[eqn]$ of the following functional equation:

$$[eqn]$$

for all $[eqn]$ , where $[eqn]$ and the functional equation Eq (1) appears in mathematical biology and the theory of learning to observe the nature of predator animals that hunt two kinds of prey. Such a conduct is defined by the Markov process in the state space [0,1] with the probabilities of transition operators g1 and g2 are given by

$[eqn]$ .

In the mathematical model Eq (1), the solution f is the final probability of the event when the predator is fixed on one category of prey, knowing that the initial probability for this category to be chosen is equal to x [14]. Also, Turab and Sintunavarat [15] observe the behaviour of the paradise fish in a two-choice situation [16].

In [12], Lyubich and Shapiro used Schauder’s fixed point theorem to prove the existence of a solution of the functional equation Eq (1) of the following form:

$$[eqn]$$

satisfying the conditions $[eqn]$ .

After this, Istra’tescu [17] proposed the existence and uniqueness result for the solution of the functional equation Eq (1) with condition Eq (2) using the Banach contraction mapping principle.

In this context, Dmitriev and Shapiro [18] found a solution of Eq (1) by a direct method. They denoted $[eqn]$ and $[eqn]$ and used the substitution

$$[eqn]$$

to reduce the functional equation Eq (3) with the unknown function f to the following functional equation:

$$[eqn]$$

where h is an unknown mapping. Thus, they proved that the solution of the functional equation Eq (4) can be presented as

$$[eqn]$$

where

$$[eqn]$$ $$[eqn]$$

and

$$[eqn]$$

Recently, the result of Istra’tescu was expanded by Berinde and Khan [19], who discussed the existence and uniqueness of a solution of the proposed functional equation using the Banach fixed point theorem. They modeled the functional equation Eq (1) in the following form:

$$[eqn]$$

where $[eqn]$ is an unknown mapping, $[eqn]$ are contraction mappings satisfying

$$[eqn]$$

In recent times, Turab et al. [15,20–25] introduced new conditions and proved the existence and uniqueness of a solution of the functional equation Eq (6) using one boundary condition in Eq (7).

This paper addresses an important open problem in [25] by applying the Matkowski fixed point theorem under minimal assumptions to a new modified functional equation Eq (8). Equation Eq (8) generalizes Eq (6) to an integral form; in particular, by choosing $[eqn]$ , $[eqn]$ , $[eqn]$ , $[eqn]$ , and evaluating at t = x, equation Eq (8) reduces to Eq (6). This shows that the earlier equation is a special case of our generalization, which we then proceed to solve.

Furthermore, we provide a solution to the open problem posed in [25] by proving that, unlike the previous setting, the initial conditions $[eqn]$ and $[eqn]$ are not essential for ensuring the existence and uniqueness of solutions. In addition, we present two experiments based on operant-control theory together with illustrative examples.

A key contribution of this work is the generalization of previous functional equations in operant-control models by formulating an integral version and applying the Matkowski fixed point theorem. This approach establishes existence and uniqueness without requiring initial conditions, demonstrates that earlier results are special cases, and provides a broader framework for modeling operant-control behavior.

Preliminaries

Following definitions and well-known fixed point results will be required in the continuation.

Definition 1. Let X and Y be two nonempty sets and $[eqn]$ be a single valued mapping. A point $[eqn]$ is called a fixed point of T if and only if x = Tx.

Definition 2. Let ( $[eqn]$ ) be a metric space. A mapping $[eqn]$ is called a Banach contraction mapping if there is a non-negative real number $[eqn]$ such that for all $[eqn]$ ,

$$[eqn]$$

Theorem 1 ( Matkowski fixed point theorem). Let ( $[eqn]$ ) be a complete metric space and suppose $[eqn]$ satisfies for all $[eqn]$ ,

$$[eqn]$$

where $[eqn]$ is non-decreasing and right-continuous such that $[eqn]$ for all t>0. Then T has a unique fixed point $[eqn]$ , i.e., there exists a unique $[eqn]$ such that $[eqn]$ .

Moreover, for any $[eqn]$ , the sequence {T ^ n ^ x} converges to the fixed point $[eqn]$ .

Fixed point theory is instrumental in solving functional equations due to its ability to transform complex problems into simpler ones by identifying points where a mapping maps to itself. This theory is key in several applications, such as iterative methods like the Picard iteration, which use fixed points to find solutions to differential and integral equations. Fixed point theorems often serve as existence proofs for solutions to various functional equations, and many problems in nonlinear analysis and differential equations rely on fixed point results to establish the existence of solutions.

Main results

Let $[eqn]$ be a collection of all continuous real-valued mappings $[eqn]$ such that $[eqn]$

If $[eqn]$ is defined by $[eqn]$ for all $[eqn]$ , then $[eqn]$ is a Banach space. Indeed, completeness follows because $[eqn]$ is a closed subspace of the classical Banach space $[eqn]$ , since the condition $[eqn]$ defines a closed subset.

Furthermore, we shall be interested in the existence and uniqueness of a solution to the following functional equation:

$$[eqn]$$

where $[eqn]$ and $[eqn]$ are non-negative and continuous mappings such that

$$[eqn]$$

$[eqn]$ and $[eqn]$ are continuous mappings from [0,1] to [0,1], and $[eqn]$ is the unknown mapping with the initial condition $[eqn]$ .

We now turn to our main results in this paper.

Theorem 2. Consider the functional equation Eq (8). Suppose that $[eqn]$ are Banach Contraction mappings with contraction coefficients $[eqn]$ respectively and satisfy $[eqn]$ Then Eq (8) has a unique solution. Moreover, the sequence $[eqn]$ in $[eqn]$ defined for each $[eqn]$ by

$$[eqn]$$

for all $[eqn]$ , where $[eqn]$ is given in $[eqn]$ , converges to a unique solution of Eq (8).

proof. Define an operator $[eqn]$ by

$$[eqn]$$

Here, the mapping T maps any continuous mapping $[eqn]$ to a new mapping $[eqn]$ defined by the integral above.

Since $[eqn]$ and $[eqn]$ are continuous mappings from [0,1] to [0,1], and $[eqn]$ is continuous on [0,1], the compositions

$$[eqn]$$

are also continuous on [0,1]. Hence, the integrand

$$[eqn]$$

is continuous in both variables x and t on the region $[eqn]$ . Therefore, the integral expression defining $[eqn]$ is continuous with respect to x, and in particular,

$$[eqn]$$

Thus, $[eqn]$ , and so T is a well-defined operator on $[eqn]$ .

Next, we show that T satisfies the conditions of the Matkowski fixed point theorem. Define a mapping $[eqn]$ by

$$[eqn]$$

where $[eqn]$ and $[eqn]$ are the contraction constants of the mappings $[eqn]$ and $[eqn]$ respectively.

Clearly, the mapping $[eqn]$ is non-decreasing and right-continuous.

Consider two mappings $[eqn]$ . For any $[eqn]$ , we have

$$[eqn]$$

Using the triangle inequality, this can be bounded by

$$[eqn]$$

Since $[eqn]$ and $[eqn]$ are Banach contraction mappings with constants $[eqn]$ and $[eqn]$ ,

$$[eqn]$$

and

$$[eqn]$$

Therefore,

$$[eqn]$$

This simplifies to

$$[eqn]$$

Taking the supremum over $[eqn]$ ,

$$[eqn]$$

Thus, the operator T satisfies the contraction condition with respect to the function $[eqn]$ .

Since $[eqn]$ , we have $[eqn]$ for all t > 0. This shows that the mapping $[eqn]$ satisfies the condition for the Matkowski Fixed Point Theorem (1).

By the Matkowski fixed point theorem, there exists a unique mapping $[eqn]$ such that $[eqn]$ . This $[eqn]$ is the unique solution to the functional equation Eq (8).

Note. The choice of Matkowski’s fixed point theorem is essential because the operator in (8), due to its integral form and the dependence of $[eqn]$ and $[eqn]$ on both x and t, does not generally satisfy the uniform contraction requirement k < 1 required by the Banach contraction principle. Matkowski’s theorem allows weaker contractive conditions through a comparison function $[eqn]$ with $[eqn]$ as $[eqn]$ , enabling us to establish existence and uniqueness where classical contraction methods fail.

Corollary 1. Consider the functional equation Eq (8). Suppose that $[eqn]$ are Banach Contraction mappings with contraction coefficients $[eqn]$ with $[eqn]$ and satisfy $[eqn]$ Then Eq (8) has a unique solution. Moreover, the sequence $[eqn]$ in $[eqn]$ defined for each $[eqn]$ by

$$[eqn]$$

for all $[eqn]$ , where $[eqn]$ is given in $[eqn]$ , converges to a unique solution of Eq 8.

The previously mentioned mathematical model is utilized to evaluate the training of rewards, addressing the limited challenges of reinforcement, routine instruction, rapid improvement within operant-control theory. In the next section, we will focus on specific experiments.

Background and motivation

The following two behavioral experiments strongly motivate our modelling framework by demonstrating that learning in operant-control systems is an adaptive, iterative process where response probabilities evolve continuously with experience. Skinner’s pigeon experiment and the human fMRI reward study both show that behavior is dynamically reshaped by reward history and stimulus predictability, revealing probabilistic adjustments far more complex than those captured by traditional discrete reinforcement models. These empirical insights justify the need for our integral functional equation approach, which models probability evolution as a continuous process and supports rigorous analysis via Matkowski’s fixed point theorem.

Experiment I: Pigeon experiment

In 1943, behavioral psychologist Skinner embarked on a seemingly far-fetched idea: using trained pigeons to guide armed missiles toward enemy targets [26]. At that time, radar had not been invented, and the mechanical equipment needed to guide a missile left little room for explosives.

Skinner [27] developed a technique known as shaping. He would provide food rewards to his subjects when they approximated the desired behavior. Gradually, as they moved closer to the goal, Skinner adjusted the rewards to be more specific, eventually guiding the subjects to perform exactly as he intended.

He began this project by showing pigeons a dot projected on a translucent screen. Initially, a pigeon approaching the screen was rewarded with food. Over time, Skinner rewarded the bird for pecking closer and closer to the dot until it was precisely pecking on the target. Eventually, he moved the dot back and forth, and the pigeon learned to follow it quickly and accurately, ultimately training to track a moving battleship. Here below, Fig 1 illustrates the experimental setup.

Pigeon project chamber.

It performed exceptionally well in tests for the Navy. The pigeons accurately guided the missile, demonstrating the effectiveness of this innovative guidance system.

In this experiment, the event that changes the probability is assumed to correspond perfectly with the pigeon’s response. We assume that, at any given time during the experiment, the probabilities of the pigeon pecking and not pecking the dot are x and 1–x, respectively. It is further assumed that, in the next trial, either response will alter the probability of pecking. This modification is modeled by continuous mappings on [0,1]. When the pigeon pecks the dot, the probability in the next trial changes at the rate $[eqn]$ , whereas if it does not peck the dot, the probability changes at the rate $[eqn]$ .

It can be described by the following functional equation:

$$[eqn]$$

for all $[eqn]$ , where $[eqn]$ and $[eqn]$ are continuous mappings from [0,1] to [0,1], and $[eqn]$ is the unknown mapping with the initial condition $[eqn]$ .

In this model, $[eqn]$ represents the probability iteration mapping, which describes how the probability of the pigeon pecking evolves over successive trials. The condition $[eqn]$ reflects the assumption that at the beginning of the experiment (before any trial is conducted), the initial probability of pecking is zero.

Experiment II : The predictability of rewards influences human behavior

An adult participant underwent fMRI scanning while receiving small amounts of either fruit juice or water orally [28]. During the scan, the subject received these liquids in either a predictable or unpredictable manner across two separate sessions.

In the predictable session, fruit juice and water were alternated every 10 seconds at a fixed interval. In the unpredictable session, the order of fruit juice and water was randomized, and the interval between stimuli was also randomized based on a Poisson distribution with an average of 10 seconds [29]. Here below, Fig 2 illustrates the design of an experiment.

Design of fMRI experiment.

The subject received 0.8 ml of both fruit juice and water through two plastic tubes. A mouthpiece held the tubes in place over the tongue, with fruit juice delivered on the left side and water on the right side.

Four conditions representing the four possible events are given by the following Table 1. Here, by the work discussed in [7], we propose the following general equation:

$$[eqn]$$

Table 1: Corresponding probabilities and operators.

for all $[eqn]$ , $[eqn]$ , where $[eqn]$ , $[eqn]$ , $[eqn]$ and $[eqn]$ are continuous mappings from [0,1] to [0,1], and $[eqn]$ is the unknown mapping with the initial condition $[eqn]$ .

In this model, $[eqn]$ denotes the probability of receiving the preferred liquid over successive trials. The initial condition $[eqn]$ reflects the assumption that, at the start of the experiment (before any trial is conducted), the probability of obtaining the preferred liquid is zero.

Remark 1. Note that the boundary value $[eqn]$ holds automatically for the integral equation. Thus, unlike the discrete form in [25] (where conditions such as $[eqn]$ are often imposed to guarantee $[eqn]$ ), no extra initial conditions on the values $[eqn]$ are needed here.

Remark 2. In the particular form (10), the weights of the two responses (t and 1–t) depend only on t, meaning the contribution of $[eqn]$ and $[eqn]$ to the integral changes linearly with t.

In the generalized form Eq (8), $[eqn]$ and $[eqn]$ are allowed to depend on both x and t, not just t. This means the weighting between the two responses can vary with respect to the current state x as well as the trial variable t.

Thus, Eq (8) generalizes Eq (11) by introducing more flexible and realistic weighting mappings that can capture more complex behavioral or probabilistic dependencies between x and t.

Illustrative examples

Next, we present two illustrative examples that demonstrate the validity of the hypotheses and the practical utility of our results.

Example 1. Consider the functional equation

$$[eqn]$$

for all $[eqn]$ and define the mappings $[eqn]$ and $[eqn]$ as follows:

$$[eqn]$$

These mappings satisfy $[eqn]$ and $[eqn]$ , so $[eqn]$ and $[eqn]$ .

Also, both $[eqn]$ and $[eqn]$ are Banach contractions mappings, Since $[eqn]$ , we have

$$[eqn]$$

So the contraction coefficient is $[eqn]$ .

For $[eqn]$ , we have

$$[eqn]$$

So the contraction coefficient is $[eqn]$ .

The sum of the contraction coefficients is:

$$[eqn]$$

Thus, the condition $[eqn]$ is satisfied. Since all the assumptions of Theorem 2 hold, the functional equation Eq (12) has a unique solution.

We define the sequence $[eqn]$ starting with $[eqn]$ and compute each successive iteration using the recursive formula

$$[eqn]$$

Since $[eqn]$ , for the first iteration $[eqn]$ , we have

$$[eqn]$$

This simplifies to

$$[eqn]$$

Breaking this down further, we compute the terms

$$[eqn]$$

Therefore

$$[eqn]$$

Simplifying further and computing this integral gives the first iteration result.

$$[eqn]$$

Next, for the second iteration $[eqn]$ , we use the recursive formula

$$[eqn]$$

Substitute the expression for $[eqn]$ into this equation and compute the integral.

$$[eqn]$$

Finally, for the third iteration $[eqn]$ , we have

$$[eqn]$$ $$[eqn]$$

After three iterations, we observe that the sequence $[eqn]$ is converging towards a fixed point mapping. The sequence continues to get closer to the unique solution of the functional equation as n increases, demonstrating the convergence guaranteed by Theorem 2. In the following, Fig 3 illustrates the first three iterations.

Sequential evolution of the iterative process.Illustration of the Picard iterative scheme applied to the operator T. (A): Initial approximation Γ0. (B): Picard iterate Γ1=TΓ0. (C): Picard iterate Γ2=TΓ1. (D): Picard iterate Γ3=TΓ2.

Example 2. Consider the functional equation defined as follows:

$$[eqn]$$

Let $[eqn]$ be defined as follows:

$$[eqn]$$

Each of these mappings does not vanish at t = 0

$$[eqn]$$

To determine the contraction coefficients, we compute

$$[eqn]$$

Thus, the contraction coefficient for each mapping $[eqn]$ ,

$$[eqn]$$

Summing the contraction coefficients yields

$$[eqn]$$

Since $[eqn]$ and the total sum of the contraction coefficients is less than 1, the functional equation Eq (13) has a unique solution.

Let the initial mapping $[eqn]$ be given by

$$[eqn]$$

Given the iterative formula

$$[eqn]$$

Now, substitute $[eqn]$

$$[eqn]$$

And the iteration formula becomes

$$[eqn]$$

Since $[eqn]$ is the same for all terms,

$$[eqn]$$ $$[eqn]$$ $$[eqn]$$

Starting with $[eqn]$

$$[eqn]$$

Calculate the integral

$$[eqn]$$ $$[eqn]$$

We compute $[eqn]$ using the iterative formula

$$[eqn]$$

Simplifying the terms

$$[eqn]$$ $$[eqn]$$

Now, for $[eqn]$ , we use the iterative formula

$$[eqn]$$

Substitute $[eqn]$

$$[eqn]$$ $$[eqn]$$

We can continue expanding and integrating each term similarly to how it was done in the second iteration. In the following, Fig 4 illustrates the first two iterations.

Sequential progression of Picard iterations demonstrating convergence toward the unique fixed point of T.(A): Initial approximation Γ0. (B): Picard iterate Γ1=TΓ0. (C): Picard iterate Γ2=TΓ1.

Conclusion

In this work, we presented a generalized functional equation Eq (8) inspired by operant-control experiments and established the existence and uniqueness of its solution using the Matkowski fixed point theorem under minimal assumptions. Our formulation extends earlier models based on behavioral probability learning and demonstrates that the integral form naturally encompasses previous discrete formulations as special cases. This theoretical advancement not only provides a stronger analytical foundation for studying adaptive and probabilistic behaviors but also broadens the applicability of functional equations in modeling complex dynamical systems. The approach highlights the versatility of fixed point theory as a unifying mathematical framework for problems encountered in behavioral sciences, control systems, and reinforcement-based learning models in engineering.

Supporting information

S1 FileMinimal dataset for iterations $[eqn]$ to $[eqn]$ .This CSV file contains the x-values in the first column and the corresponding values of $[eqn]$ , $[eqn]$ , $[eqn]$ , and $[eqn]$ in subsequent columns. It allows replication of all iterative results and Figs 3 and 4 presented in the manuscript. Columns are labeled: x, Gamma0, Gamma1, Gamma2, Gamma3.(XLSX)