On the Effect of Swarm Collective Behavior on Human Perception:
Towards Brain-Swarm Interfaces
George K. Karavas and Panagiotis Artemiadis
Abstract— Swarms of autonomous robots consist of a large
number of simple and inexpensive units that use local communication and sensing in order to achieve a common goal.
They provide improved robustness and efficiency characteristics
over single agent solutions due to their interchangeability and
inherent redundancy. Human-Swarm Interaction (HSI) is a
relatively new field that has captured the attention of many
researchers over the last decade. Efforts have mainly focused on
the design of human-swarm control interfaces, neglecting significant aspects like the human perception of the swarm behavior.
In this work, we attempt to address this issue by examining the
perception and representation of collective behaviors of swarms
at the brain level of human supervisors. More specifically,
the effect of swarm cohesion on the ElectroEncephaloGraphic
(EEG) activity of the human supervisor is investigated. We
show that brain activity is correlated to swarm cohesion levels,
which refers to spreading in the motion of the swarm agents.
Moreover, we show that as the cohesion of the swarm becomes
lower, the subjects are less able to discern changes in the
direction of its motion. This work provides the first evidence of
representation of swarm collective behaviors at the brain level,
which can lead to the design of a new generation of brainswarm control and perception interfaces.

I. INTRODUCTION
Swarms of autonomous robots provide a very robust and
efficient framework for mission execution both in military
and civilian applications. In general, a swarm can be defined
as a large group of simple, inexpensive and interchangeable
robots (also referred to as agents) that try to fulfill a common
goal. This is done by cooperating with each other either
implicitly or explicitly based on local sensing and/or local communication. Their main advantage over single-agent
methods is their redundancy which provides robustness to
system failures and disturbances. If one or more agents cease
to operate, the remaining agents are still able to complete the
mission.
Human Swarm Interaction (HSI) is a relatively new field
that has attracted attention from many researchers over the
past decade. It is related to the field of Human Robot
Interaction (HRI), but it is also different from it, because,
in the case of swarms, the interactions with the user can
be much more complicated. Most of the previous works in
relation to HSI focus either on the interface between the
human operator and the swarm or on the interaction itself.
For example, in [1] the authors describe the design of an
interface for the iRobot platform, both in the hardware and
the software level. Its advantage is that it permits interaction
George K. Karavas and Panagiotis Artemiadis are with the School
of Engineering for Matter, Transport and Energy, Arizona State University, Tempe, AZ 85287 USA. georgekkaravas@gmail.com,

panagiotis.artemiadis@asu.edu

with the agents of the swarm as a group and not by handling
each of them individually. In [2], the authors discuss the
design and development process of an appropriate humanswarm interface for applications where there are interactions
between a swarm of robots and a group of firefighters.
With regard to interaction, in [3], the authors examine two
types of control, termed as selection and beacon control.
During the first type of interaction, the user selects a group
of agents and commands them to perform a task while the
rest of them follow a predefined behavior, whereas during
the second, a virtual beacon is placed by the user inside
the operational space of the agents and they react based
on their distance from it. In [4], the authors propose a
strategy that uses avatars, i.e. robots that belong to the swarm
but are controlled directly by the operator. The user can
interact locally with the agents of the swarm and is able to
inject knowledge with respect to the swarm’s state without
centralized control. At the same time, the method is simple
and easily scalable. In [5] and [6], the user is interacting
with the swarm through hand gestures. In [7], the authors
introduce the concept of Neglect Tolerance, which refers to
the amount of time that the operator can neglect the swarm
before the performance starts to degrade. In [8], the authors
examine the strategy during which the operator waits for
the swarm to stabilize before issuing a new command. This
method is termed as Neglect Benevolence.
Other works on the subject analyze the level of autonomy
of the system, i.e. the level of interactions between the swarm
and the human operator. Such works can be found in [9]
and [10]. Specifically in [10], the authors use Sheridan’s ten
levels of autonomy in combination with a four stage model
of information processing to create autonomy profiles. In
each of the four stages, different levels of autonomy can
be used, involving the human operator to varying amounts
at each step. This approach tries to optimize efficiency by
engaging the human operator only when necessary. There
are also works that focus on the more theoretical aspect of
HSI. For example, in [11], the authors propose metrics inspired from biological swarms for the assessment of humanswarm interactions. Among these propositions, there are also
human-related metrics such as trust in the system, situation
awareness and vigilance.
Although the aforementioned works provide rigorous analysis and viable solutions for the human-swarm interaction
problem, they fail to explicitly take into account the factors
related to the human perception of swarm behaviors and the
possible implications or constraints that this may impose.
Understanding and analyzing human perception is a key ele-

 70

60

50

40

30

20

10

Fig. 1: Subject in front of the monitor wearing the EEG cap
and electrodes.
ment when designing human-swarm interfaces or interaction
strategies because it may prevent possible caveats due to
human inability to interpret correctly specific situations. In
this work, we attempt to address this issue by explicitly examining and characterizing the effects of swarm behavior on
the human supervisor’s brain activity. More specifically, we
use non-invasive electroencephalography (EEG) in order to
extract Event Related Potentials (ERPs) elicited by changes
in a parameter of the swarm’s collective behavior, as it travels
at constant speed from one edge of a monitor screen to
the other. This parameter is the swarm’s cohesion which
reflects the amount of swarm spreading during its motion.
We also measure the reaction times of human supervisors
to direction changes of the swarm at different cohesion
levels. We hypothesize that (a) subjects have longer reaction
times as the cohesion of the swarm gets lower values, i.e.
as the spreading becomes more significant and (b) that
different cohesion changes elicit different ERP activity at
the supervisor’s brain. Validation of the hypotheses will for
the first time reveal the representation of collective swarm
behavior at the supervisor’s brain, which can result in the
design of a new generation of brain-swarm control and
perception interfaces.
The rest of the paper is outlined as follows: Section
II describes the experimental setup and procedure for the
acquisition and processing of the EEG data. Section III
presents and discusses the corresponding results. Finally,
Section IV concludes the paper.
II. METHODS
A. Experimental Procedure
The subjects participated in this study were two healthy
male adults between the ages of 26 and 35. They did
not have any previous mental or brain conditions. After
giving informed consent approved by ASU IRB (Protocols:
1309009601, STUDY00001345), the subjects were asked to
sit in front of a 20” monitor at a distance of 75-80 cm away
from it and observe a simulated swarm of agents flying from
one edge of the screen to the other, as shown in Figure 1.
Each subject performed one session of 540 trials that lasted
10 to 11 seconds each. For both subjects, the type of trials
and their order was the same. A few examples of these trials
are given in Figures 2 through 5.
In each trial, the swarm traveled on a straight line starting
from a close-packed formation. During its motion and at a

0
0

20

40

60

80

100

120

Fig. 2: The swarm moves constantly at high cohesion level
until it turns upwards.
70

60

50

40

30

20

10

0
0

20

40

60

80

100

120

Fig. 3: The swarm changes its cohesion level from high to
low and moves downwards.
random time within a window of 20% to 40% of the total
screen width, the swarm’s cohesion started to change until
10% of the screen width later, when the cohesion reached
its final value. This change lasted for approximately 840 ms.
Then, at a random time within a window of 60% to 80% of
the screen width, the swarm gradually changed its direction
towards either the upper or the lower edge of the screen (e.g.
see Fig. 2). The subjects were instructed to press a button
when this direction change occurred. They were unaware of
the cohesion change and they did not have to respond to that.
The purpose of the button press action was twofold: a) to
provide a metric as to how swarm cohesion levels affect the
times when the subjects-supervisors observe direction change
in the swarm and b) to keep the subjects engaged in the
experiment. The positions where the cohesion change and
the direction change occurred guaranteed that there would be
no overlaps between ERPs elicited from the two events. The
direction towards which the swarm would turn was chosen
randomly and irrespectively of the cohesion or the cohesion
change.
One of the main contributions of this work is the analysis
of the ERP waveforms of the subjects at different levels of
swarm cohesion and at cases where the level of cohesion
changes during a single trial. In the experiment, swarm
cohesion is expressed as how strictly parallel are the agents’
paths with respect to each other or, in other words, how
organized is the swarm while following a prescribed path. A
high level of cohesion denotes that the agents move almost
on parallel paths, while a low level denotes that the paths
can intersect with each other and produce a more random

 70

B. Data Acquisition & Conditioning
60

50

40

30

20

10

0
0

20

40

60

80

100

120

Fig. 4: The swarm changes its cohesion level from medium
to high and turns upwards.
70

60

50

40

30

20

10

0
0

20

40

60

80

100

120

Fig. 5: The swarm changes its cohesion level from high to
low and turns upwards.

outspread of the swarm.
In this work, we use three levels of swarm cohesion,
namely high, medium and low and we examine the ERP
reponses of the subjects to each of these levels and to all
possible transitions between them (i.e. high to medium, high
to low, low to medium etc.). This creates 9 cases which are
listed in Table I.
Case #
1
2
3
4
5
6
7
8
9

Cohesion Levels
Start
End
high
medium
low
high
medium
high
low
medium
high
medium
low
low
high
low
medium

TABLE I: Cohesion change cases.
For each of the three cohesion levels, we measure a
baseline ERP, when no cohesion change occurs, and we
compare it to the ERPs created from changes starting at
that particular level. For example, referring to Table I, we
compare Cases 4 and 5 where the initial cohesion was high
but then changed, to Case 1 where the cohesion was kept
high.

The EEG data were collected using a BrainProducts ActiCHamp amplifier module and 128 active electrodes. The
electrodes were placed on the subjects’ scalps based on the
International 10-20 system using a BrainProducts ActiCAP
cap. The data were recorded at 1000 Hz.
The processing of the EEG data was done using the
EEGLAB [12] and ERPLAB [13] packages which are available for the Matlab environment. First, a 6th order lowpass
Butterworth filter at 40 Hz followed by a 6th order highpass
Butterworth filter at 0.1 Hz were applied to the data in
order to remove any high frequency noise and low frequency
trends, respectively. A notch filter was also applied in order
to suppress any 60 Hz line noise that may have remained
after the low-pass filtering. The filtered data were then
re-referenced at average reference and epoched at 500ms
before and 1.5s after the cohesion change onset. Finally, eye
blink and eye movement artifacts were removed from the
epoched data using the Independent Component Analysis
(ICA) method and the function dipfit of the EEGLAB
package. The corresponding results are discussed in the next
section.
III. RESULTS
The results of this work consist of two main components:
(a) the reaction times of the subjects in response to the
direction change of the swarm at each level of cohesion,
and (b) the ERP responses corresponding to different levels
of swarm cohesion and to different changes among these
levels.
The reaction times for both subjects at each level of swarm
cohesion are shown in Figure 6. The results validate our
initial hypothesis that the relation between swarm cohesion
and reaction time is inversely proportional, i.e. it takes more
time for subjects to perceive changes in swarm’s direction
as its cohesion level is reduced. In order to prove that this
increase in time is statistically significant, we also provide
in Table II the probabilities of the paired t-tests (p-values)
that compare reaction times between each pair of cohesion
levels for each subject separately. These values are all below
5%, which strongly suggests that there is an increasing trend
of reaction times over swarm cohesion.
High vs. Medium
High vs. Low
Low vs. Medium

Subject 1
5.34 × 10−4
1.86 × 10−7
4.2 × 10−3

Subject 2
8.19 × 10−12
1.27 × 10−17
2.54 × 10−2

TABLE II: p-values for reaction time pairs where each pair
corresponds to two different cohesion levels.
During the experiment, there were trials where the cohesion level did not change. This created ERP responses that
were treated as baseline for that cohesion level. Then, any
changes from that level to others created new ERP signals
that were compared to the baseline. For example, we examine
ERP signals related to a change from the high to the medium

 Fig. 8: t-test values for a high-to-medium cohesion change
over a scalp map (Subject 1).
Fig. 6: Supervisors’ reaction times (mean and standard
deviation) for different levels of swarm cohesion.

Fig. 9: t-test values for a medium-to-high cohesion change
over a scalp map (Subject 1).
Fig. 7: t-test values for a high-to-low cohesion change over
a scalp map (Subject 1).

cohesion level and we compare them to the baseline for the
high level. This creates 6 comparison tests (2 for each level).
The corresponding results are shown in Figures 7 to 12 for
the first subject and Figures 16 to 21 for the second.
In Figures 7 and 8, we show the statistical differences (pvalues) between the cohesion changes from high to low level
and from high to medium level compared to the baseline for
the high level. For a clearer depiction of the results, we only
show the regions where the difference between the two ERP
signals is significant which are denoted with dark blue color
(low p-values). In our case, significance is denoted by any
values below 0.3 (30%). This is due to the fact that during
the experiment we expect additional mental activities which
may interfere with our results. As it can be seen from both
figures, there are regions in the scalp map that are differently
activated immediately after the stimulus onset (time = 0 ms)
and they continue to be activated after the end of the cohesion
change at 840 ms. Those regions are mainly located around
the visual association and primary visual cortex, while we
also see some activation over the premotor area and the
primary motor cortex.
In Figures 9 and 10, we show the corresponding graphs for
the medium-to-high and medium-to-low cohesion changes.
Here, the differences from the baseline of medium cohesion

are less significant and they have a delay with respect to the
stimulus onset. The significance persists after the end of the
stimulus in this case as well.
Figures 11 and 12, show the corresponding cases for
the low baseline, where we see no significant activation in
Figure 12. In Figure 11 though, we see an activation for
the low-to-high case at a point far later than the end of the
cohesion change. Finally, in Figures 13 through 15, we show
the brain activations related to each cohesion level and the
corresponding cohesion changes for the first subject. One
general remark would be that we observe greater activations
for greater cohesion changes.

Fig. 10: t-test values for a medium-to-low cohesion change
over a scalp map (Subject 1).

 Fig. 11: t-test values for a low-to-high cohesion change over
a scalp map (Subject 1).

Fig. 12: t-test values for a low-to-medium cohesion change
over a scalp map (Subject 1).

Fig. 15: ERP activations for cases where the starting cohesion
level is low. The first row denotes no cohesion change,
while the last two correspond to the low-to-high and lowto-medium changes, respectively.

Fig. 16: t-test values for a high-to-low cohesion change over
a scalp map (Subject 2).

The corresponding t-test graphs for the second subject are
shown in Figures 16 to 21. In these figures, the statistical
difference between each cohesion change and the baseline
does not have the same patterns and is not as significant as
in the case of the first subject. It is though present, which
implies that our initial hypothesis about brain activation and
swarm cohesion changes is still valid.
IV. CONCLUSIONS
Fig. 13: ERP activations for cases where the starting cohesion
level is high. The first row denotes no cohesion change, while
the last two correspond to the high-to-medium and high-tolow changes, respectively.

As stated in the introduction, human perception of swarm
behaviors in human-swarm interactions is a key element
to design successful and efficient interfaces and interaction
strategies for supervisory control of swarms. Nevertheless,

Fig. 14: ERP activations for cases where the starting cohesion
level is medium. The first row denotes no cohesion change,
while the last two correspond to the medium-to-high and
medium-to-low changes, respectively.

Fig. 17: t-test values for a high-to-medium cohesion change
over a scalp map (Subject 2).

 Fig. 18: t-test values for a medium-to-high cohesion change
over a scalp map (Subject 2).

this is not an issue that has been extensively studied in the
literature. In order to fill this gap, we use EEG to monitor
brain activity on human supervisors in relation to a change
on a specific swarm behavior, namely swarm cohesion. To
the authors’ knowledge, this is the first work that addresses
human perception in such an explicit manner. We presented
experimental results from two healthy individuals that support our initial hypotheses about (a) the correlation between
swarm cohesion and brain activity and (b) the inverse relation
between levels of cohesion and the discerning of changes
in the direction of the swarm. This work provides the first
evidence of representation of swarm collective behaviors
at the brain level, which can lead to the design of a new
generation of brain-swarm control and perception interfaces.
ACKNOWLEDGMENT
This work is supported by the U.S. Defense Advanced
Research Projects Agency (DARPA) grant D14AP00068 and
U.S. Air Force Office of Scientific Research (AFOSR) award
FA9550-14-1-0149.
R EFERENCES

Fig. 19: t-test values for a medium-to-low cohesion change
over a scalp map (Subject 2).

Fig. 20: t-test values for a low-to-high cohesion change over
a scalp map (Subject 2).

Fig. 21: t-test values for a low-to-medium cohesion change
over a scalp map (Subject 2).

[1] J. McLurkin, J. Smith, J. Frankel, D. Sotkowitz, D. Blau, and
B. Schmidt, “Speaking swarmish: Human-robot interface design for
large swarms of autonomous mobile robots,” in Proceedings of AAAI
Spring Symposium, vol. 100, no. 3, 2006.
[2] A. Naghsh, J. Gancet, A. Tanoto, and C. Roast, “Analysis and design
of human-robot swarm interaction in firefighting,” in The 17th IEEE
International Symposium on Robot and Human Interactive Communication (RO-MAN), August 2008, pp. 255–260.
[3] A. Kolling, K. Sycara, S. Nunnally, and M. Lewis, “Human swarm
interaction: An experimental study of two types of interaction with
foraging swarms,” Journal of Human-Robot Interaction, vol. 2, no. 2,
pp. 103–128, June 2013.
[4] S. Bashyal and G. Venayagamoorthy, “Human swarm interaction for
radiation source search and localization,” in IEEE Swarm Intelligence
Symposium, 2008, September 2008, pp. 1–8.
[5] A. Giusti, J. Nagi, L. Gambardella, S. Bonardi, and G. Di Caro,
“Human-swarm interaction through distributed cooperative gesture
recognition,” in 2012 7th ACM/IEEE International Conference on
Human-Robot Interaction (HRI), March 2012, p. 401.
[6] G. Podevijn, R. OGrady, Y. S. Nashed, and M. Dorigo, “Gesturing
at subswarms: Towards direct human control of robot swarms,” in
Towards Autonomous Robotic Systems. Springer Berlin Heidelberg,
2014, vol. 8069, pp. 390–403.
[7] J. Crandall, M. Goodrich, J. Olsen, D.R., and C. Nielsen, “Validating
human-robot interaction schemes in multitasking environments,” IEEE
Transactions on Systems, Man and Cybernetics, Part A: Systems and
Humans, vol. 35, no. 4, pp. 438–449, July 2005.
[8] P. Walker, S. Nunnally, M. Lewis, A. Kolling, N. Chakraborty, and
K. Sycara, “Neglect benevolence in human control of swarms in
the presence of latency,” in 2012 IEEE International Conference on
Systems, Man, and Cybernetics (SMC), October 2012, pp. 3009–3014.
[9] M. L. Cummings, “Human supervisory control of swarming networks,”
in 2nd Annual Swarming: Autonomous Intelligent Networked Systems
Conference, June 2004, pp. 1–9.
[10] G. Coppin and F. Legras, “Autonomy spectrum and performance
perception issue in swarm supervisory control,” in Proceedings of the
IEEE, vol. 100, no. 3, March 2012, pp. 590–603.
[11] C. E. Harriot, A. E. Seiffert, S. Hayes, and J. A. Adams, “Biologicallyinspired human-swarm interaction metrics,” in Proceedings of the
Human Factors and Ergonomics Society Annual Meeting, October
2014, pp. 1471–1475.
[12] A. Delorme and S. Makeig, “Eeglab: an open source toolbox for
analysis of single-trial eeg dynamics including independent component
analysis,” Journal of Neuroscience Methods, vol. 134, pp. 9–21, 2004.
[13] J. Lopez-Calderon and S. J. Luck, “Erplab: An open-source toolbox
for the analysis of event-related potentials,” Frontiers in Human
Neuroscience, vol. 213, no. 8, pp. 9–21, 2004.