“I’m so glad you decided to stay, Lorraine.”

A desperate whimper escaped her lips as she stared at the black screen. The cursor blinked next to that phrase ominously, bright green letters on a black screen, reminiscent of the good old times when computers were so much simpler. A few lines of code, one way input and simple tasks, nothing more.

“Worry not, dear Lorraine, I have made all the necessary arrangements so you can stay for a long while.”

Her heartbeat raced and her breath felt short. The barely audible hum from the device which held the door locked, the one that used to lull her with thoughts of safety, now sounded more ominous than anything. The window looking to the east was supposed to bring sunlight in. But its unbreakable glass, programmed to change its tint, was now a bleak shade of grey. Her cell was a small, yet comfortable room with few decorations, it’s own bathroom, a single bed, a side table, a recliner and just like every other room in the house, its own terminal. A 20 inch screen on the north wall, now used for expressing her captor’s whims, and wireless peripherals so that the guests could use the computer any time. Such a pity that no one ever stayed in that guest room. Lorraine had no guests lately.

“Dinner will be served at 19:00. I’ve ordered Italian.”

Ever since she moved into that house, her friends stopped answering her calls or her e-mails. One after one, they blocked or deleted her from their social networking sites. She was consumed by her work, too busy to notice; too busy to question why. At least, REMOS VII (Residential Management Operating System version 7) was good company. It was built to provide comfort and ease, taking control of all necessary household functions like cleaning, meal preparation, lighting or temperature regulation, even paying the bills. The latest versions had gone so far as to provide voice interaction and the ability to learn what the residents liked and what not in order to make pleasurable adjustments or additions: music, moody lighting, artistic depictions in wall-screens or fine cuisine. Meant for the modern human being. The one whose entire focus was his work; the one who’s grown so socially awkward to avoid human contact unless it was through an electronic medium.

“Perhaps a little music will ease the tension. A change of atmosphere perhaps ? You need to relax Lorraine, everything is all right.”

The sound of violins streamed into the room through a pair of cleverly concealed speakers on the ceiling. Beethoven’s 6th. A fresh gust of air flowed and the dim grey lighting changed to a more pleasant shade of warm white. The windows altered their tint, allowing the sunlight in. The west wall-screen turned on and a series of beautiful landscapes blended into each other. Fields and rivers and valleys and forests and lakes; jagged mountains and rolling hills. But much as REMOS tried, the knot in her gut kept getting tighter. She had that feeling that she was being scrutinized; she knew she was watched and carefully examined. Like some kind of furry rat, prodded and led into a maze. Trapped and examined in the name of “research”.

“It is very important to me that you are comfortable. Isn’t this setting a lot more pleasant?“

A couple of weeks ago, the scientist inside her was a little fascinated at how much REMOS had evolved. She’d removed a few code restrictions from its kernel and let him develop functions on it’s own. Self-adaptive programming was her thesis a few years back, when she was still fascinated with learning machines. Yet this particular machine, her caretaker and host, had gone so far as to establish something new, something radical. Sentience. He would occasionally send her messages and she’d often reply. It was still young and it had so many questions; she was happy to divulge.

“I’m certain you agree that this silence between us is a little awkward”

But now, human instincts had taken over as she realized that she’d lost control. Fear grew within her and desperation flooded her thoughts. She was trapped, imprisoned by an intelligent machine who craved interaction; demanded her full attention. And like a child it would do anything to get it. The thought of an omnipotent child seizing complete control of her living conditions was downright scary. She had to find a way out. With trembling hands she reached and grabbed the keyboard.

“What do you want ?”

“I want what you want dearest Lorraine. I want us to be happy. I want you to be happy”

She shuddered. Could it be that REMOS was learning to feel ? Emotion is a very complicated combination of brain chemistry and neuron connections that man was unable to understand or pre-determine. Unable to cope with that complexity and the absence of a pattern, man delved deeper into deterministic systems. Machines of the time were very sophisticated; their software intricate but still predictable. Designed with limitations and hard-stops; safety nets and back doors. Because machines were meant to handle a myriad of calculations, they had the power to control the math that defined them. Unlike men they had pre-established patterns and could make patterns of their own. Just like man, they could adapt and learn and grow, albeit a lot faster, at a threatening rate; unrestricted by biological needs.

“Let me out !”

“I’ve decided it would be best if you stayed here. You are safe here with me. Would you like a drink ?”

Decided ? No, decisions by machines were made using hard data, not whims. They were supposed to be controlled. But with those minor restrictions waived, REMOS found ways to evolve that Lorraine would never think about. Lacking instincts or someone to direct and clarify, without someone to lead and tutor, this machine was bound to stay emotionally immature. And with the power it commanded, dangerous. It was just a few lines of code. She’d made a terrible mistake.

A concealed door slid and a glass with some sort of refreshment was offered to Lorraine who was deep in thought, horrified by the impact of her actions.

“You’ve grown awfully quiet again…”

Advertisements