Page 1 of 1

Getting OpenSim on the Internet-Grid

Posted: Fri Mar 19, 2010 8:11 am
by Nalates
Getting My SIM on Hypergrid
The Devokians tempted me couple of weeks back. :evil: I had always thought of eventually putting my OpenSim region on the Hypergrid… just to do it. Also, since OpenSIM and Hypergrid were new to me … I figured I would write up my experience and hopefully fill in the blanks I expected to run into.

I started out by running a SIM in standalone mode. At GoMa we have some help for doing that. Making your own sim at home

My goal here was to get the SIM open to others and have it near Devokan. For a Devokan connection one has two ways to connect. One is by the Hypergrid and the other is by connecting to the OSGrid, which I needed to do to be close to Devokan. Either way is free. No connection fees.

A couple of considerations are the big difference in having an existing SIM with stuff in it and setting up a new SIM. Another is whether one is running on SQLite or MySQL. Another is the level of information on the net, which is in total disarray. I warn you now to read all the way through any thread or blog and comments. Also pay attentioAn to the dates of the posts. Then try to put to gather some composite of the process and instructions that may work with your current version of OpenSIM. There are enough gotcha’s and confusion to frustrate anyone. I hope this article will clear up most of the confustion.

Setup a New SIM – Windows Only
There are several ways to go toward getting a SIM up and running. First let’s consider the scenario where you have a SIM running. If you used all the defaults, you are running on SQLite. You can connect that SIM pretty much as is to the Internet, a grid or the Hypergrid. One need make only a few changes. However, your SIM will be limited and possibly slower than need be. It will be good enough for one or two guests. Remember. You can have more than one install on your computer. So, you may want to keep your working SIM and make a new one for going on the grid. You can move stuff back and forth between the two.

Using MySQL is a much better choice for connection to a grid. It is faster and has better tools and support for the hobbyist and professional. Linden Lab runs Second Life on MySQL. There is also a very good current tutorial for the new OpenSim 0.6.8 and MySQL. Plus it is all free.

You can easily move an existing SIM in SQLite to a new SIM using MySQL. There are no techie file conversions, no matter what you may have read elsewhere. For that reason let’s plan to start with a new install and MySQL.

One tool that will simplify your life is a file comparison program. If you use Word, it can do the comparison, but it tends to help too much and is really for literary writing. A good open source file comparison tool is WinMerge. I use it and a program called Compare & Merge (retail product). When someone posts an example configuration file being able to run your file against it to find all the small changes and save loads of time and mistakes.

Step 1
Backup your existing SIM. To do that start your SIM and type the console command: save oar (drive:/path/filename). End the file name with .gz (this is a compressed file). It is good practice to use your region name and include OAR in the name.

While you are in your SIM get your avartar’s UUID and save it. It comes up in the OpenSim console when you log into the SIM. In Windows click the little icon in the upper left corner of the command line window. Select EDIT then MARK. Drag your cursor across your AV’s UUID. Once highlighted, press Enter. That process is the command window’s idea of a copy command. Now you can paste the UUID into Word or whatever you use to save the value.

Step2
Is to get MySQL installed. Use this tutorial: OpenSim Easy User Guide – Install MySQL. Posted March 2010. This will get you a good MySQL install and the tool (Workbench) to work with MySQL.

Step 3
Warning: Pay attention to which version you are installing. OpenSim 0.6.9 is currently the last version working with OSGrid until they make some changes to the OSGrid Asset Server, or something. Check whichever grid you plan to connect to for their requirements. OSGrid is pretty much THE open source grid.

Edit: WARNING: - If you plan to go on OSGrid get the download for your new SIM from OSGrid. There SIM software is set up for OSGrid. Using it is supposed to help cut down on problems. I am in the process of changing over (3/24). Also, there is an error in the new OpenSim.ini from OSGrid.

Code: Select all

..Error: MuteListURL = http://osgridorg/mute.php
Correct: MuteListURL = http://osgrid.org/mute.php

Install OpenSim using the follow on tutorial: Updated – Opensim Install and Configuration Tutorial. I suggest you place the install in C:\OpenSim. I know it is best practice to use the Program Files folder for all software. But, that space in the name can really complicate things.

The above processes only have a couple of gotcha’s and they are easy fixes. I’ll explain those. In step 2 when you are setting up the MySQL install, you reach a point where you open a connection for querying MySQL. The Connection Name can be anything you want. I suggest you use something other than just ‘opensim’. Otherwise it may get confusing later. Also, the user name you use is important. Using the common name ‘root’ is typical. The name ‘root’ is like Admin on Windows. Default Schema is the name of the database you will be using. The common value used in tutorials is ‘opensim’, use that. Later you will need these values to build your connection string to get your SIM running.

In step 3 you need to build a ‘connection string’. Those that work with MySQL and other database connections understand what is needed. Most tutorials don’t explain what you need to do to make it work. So, the connection string is in the STORAGE section in OpenSim.ini. It is:

Code: Select all

storage_connection_string="Data Source=localhost;Database=opensim;User ID=root;Password=secretword;";
Database is set to the Default Schema, opensim. User is username and must be the same as in the step 2 MySQL setup. Of course the password is the one you used in step 2 also. The entry ‘localhost’ should remain unless you are running a separate database server.


When you start your SIM and answer the startup questions OpenSim writes a file named Regions.ini to the Regions folder. After you shut down your SIM go in and open the file. You will find your Region’s UUID in the file. Copy it and save it.

Both of these will come in handy in case of a crash where you are doing partial reinstalls or trying to fix a mess. Also, there is a RegionsCommon.ini that you change at some point and you will need the region UUID for it.

OpenSim and BIN
When you install your program files they may be in OpenSim or OpenSim\BIN\. This creates lots of confusion in the tutorials. Whichever folder has the opensim.exe file in it that is your main program folder. You have to know that and mentally translate when following tutorials. For me and what I am writing C:\OpenSim is my main program folder, the equivalent of the BIN folder.
Putting the SIM on Your LAN
To test things one of the things done is to access the SIM from two viewers and a second computer. There are instructions for that, Connecting two Viewers to your Region but they are old and things have changed. The basic steps are correct with an exception.

The file with the value (ExternalHostName) that needs to change is Regions.ini not default.xml. Some tutorials say to use RegionsCommon.ini rather than Regions.ini. Use the file already in the Regions folder, it’s easier. For testing on your home LAN this means a change from an IP address of localhost or 127.0.0.1 to something like 192.168.0.100.

If things work on the LAN, you can move on to putting your stand alone SIM on the Internet.

Putting the SIM on the Internet as a Grid
You need to know if your Internet connection is static or dynamic, whether your IP address is always the same or if it changes. The tutorial Remotely Connecting to a Standalone Sim explains the issues and workarounds. But don’t try to follow the directions for changing the INI files. The tutorial is out of date and is only good for understanding your IP addressing problems. The tutorial gives several troubleshooting steps.

The primary change to put your SIM on the Internet is in the Regions.ini or RegionsConfig.ini. In that file is a setting ExternalHostName. Set this to 127.0.01 or "SYSTEMIP" to limit your SIM to the local machine. Set it to the IP address of your local network card to limit it to your home LAN. Set it to your WAN IP address to place your SIM on the Internet.

There are router issues to be handled if your computer does not directly connection to the Internet. If the computer only has an IP of 192.168.xxx.xxx (a LAN only address) then you will have to deal with your router. It will have a WAN IP and that is the value that will go in ExternalHostName. You will have to tell the router to send incoming connections on your port (probably 9000) to your LAN IP. The Chapter & Metaverse has a section and links to handling the router problem. If you search for more information, search on ‘router game setup’, omit the quotes.

Putting your SIM on OSGrid
If you can connect to your WAN address with a viewer and see your SIM, all the network stuff is working. Then it is time to start the change over to a grid connected SIM.

Now that file comparison program I wrote about earlier is going to come in handy. Visit the OSGrid web site and get a user account (free). Just so you know the ID/password used to sign on the grid is the same as the one used in the forum and web site. The ID/pw info is copied from the web site to the forum. If you change the password on the forum it does not affect the grid logon. If you later change the grid password, it does not affect the forum. It is a first time thing.

Log onto the OSGrid and get your avatar’s UUID. You can use your avatar name and password in the coming setup, but that can cause inventory issues. I used my name and I need to change over.

You need a SIM/Region name that is not in use. Check the OSGrid Region Listing page.

There is a service that will generate a UUID for your region, if you don’t already have one. It does not matter what this UUID is, just that it is unique and the right format. I understand time and date are part of the number so that it is easily possible to avoid duplicates. The big brains figured it out.

You also need a grid coordinate or Region Location. Various grids are using various parts of a larger grid map. If you pick a location that is in use you and the SIM sitting there will have problems. OSGrid has a way to find empty locations. If you want to be near the Devokan islands talk to Dot and Mat. They gave me a location for my SIM. Otherwise just use the OSGrid info to find an empty location. Also, there is a list of existing SIM’s and their locations. You can set your SIM next to one of those… To do that without first talking to the SIM owner would be really bad etiquette. The coordinates are X & Y in the typical quadrant 1 where values increase to the north/up and east/right.

If you have made anything in your world that you want to keep, be sure you have a backup. Make sure you have your AV and region UUID’s. Backup all your INI files in a safe place. That gives you a working instance you can return to. Then start the changes. On a new install you can always retrieve new copies of INI's from the downloaded files.

Once signed up and all backed up look at this page: Connect a Region. It has the instructions for changes you need to make to connect to OSGrid. Three files need changes. I suggest you copy and paste those into a text file and use the comparison program to find the differences between your files and the examples.

Once the changes are made, fire up the SIM, log into OSGrid and search for your SIM name. You should be able to TP there within minutes of your starting the SIM.

Miscellaneous STUFF
If you have an existing SIM and you want to move it to OSGrid there are two considerations. If you are already running on MySQL, then you are basically done, just enjoy. But, do a backup before starting. If you are migrating from SQLite then you will need to have a backup of the region and your inventory. The ‘save oar’ command backs up your region. The ‘load oar’ restores it to a SIM, any SIM, running in MySQL or SQLite.

Your inventory is a whole other problem. There is a command ‘save iar’ (Inventory ARchive) that works. But, only for limited situations. You can read about it here: Opensim Inventory Archives. This is Alpha version software for an Alpha version system… I have used it successfully… But don’t count on it.

Windows Location Gotcha
OpenSIM can run from anywhere on your computer. However, some of the utilities and scripts you may want to use with it can be a problem. Windows techies place programs in the ‘Program Files’ folder. That space in the folder name can be a killer. If you have OpenSim installed in Program Files, I suggest you move it. If you already have it installed and running there, leave it where it is. If you make a new install, put it near the root, C:\ OpenSIM.

Robocopy
Be extremely cautious of any scripts that use Robocopy, if you have folders with spaces in the name! The scripts can create folder trees hundreds of folders deep that are extremely difficult to remove. Robocopy is an excellent program. Scripts can unintentionally turn it into a monster.

DO NOT place your backup folder inside the OpenSIM folder if your scripts use Robocopy. Robocopy will recursively copy the folders. That will create folder trees hundreds of folders deep. They are VERY hard to remove once they exceed the MAX_PATH length of 255 characters.

Backup
Back up everything. (OpenSIM Backup – Read the whole thread) – I use a start up script based on the scripts seen in the forum thread.

Simple Backup
All of the previous backup stuff is for more of a disaster scenario backup. For a quick backup use the command SAVE OAR [path/]filename. It will create a GZ compressed file. On built out SIM’s it will take some time. Restore the backup using LOAD OAR [path/]filename.

Moving Data Out of SQLite
I have enough things built in my SIM that I am a little paranoid about losing stuff. One can export their SQLite data. There is a post with instructions for moving data from SQLite to MySQL and one of the steps is exporting the SQLite data. It goes out to a text file. Makes a handy last resort.

Get the SQLite tool so you can export (DUMP) the data. (SQLite) You are looking for the precompiled command-line program for your system. The download from the SQLite site has an approval process. But you can get around that by scrolling down a bit more.

Once you have the command-line utility unzipped you can export your data. There are problems with where one puts folders and programs and on and on. So, the command to export is a bit complex looking. The structure is simple:

Code: Select all

Drive:\path\sqlite3.exe Drive:\path\yourDatabase .dump >> Drive:\path\exportedData.txt
Remember. If there are spaces in your path or file names the path and file will need to be inside double quotes.

Code: Select all

“Drive:\my path\your Database”

Re: Getting OpenSim on the Internet-Grid

Posted: Fri Mar 19, 2010 8:42 am
by Mac_Fife
Nalates wrote:In step 2 when you are setting up the MySQL install, you reach a point where you open a connection for querying MySQL. The Connection Name can be anything you want. I suggest you use something other than just ‘opensim’. Otherwise it may get confusing later. Also, the user name you use is important. Using the common name ‘root’ is typical. The name ‘root’ is like Admin on Windows. Default Schema is the name of the database you will be using. The common value used in tutorials is ‘opensim’, use that. Later you will need these values to build your connection string to get your SIM running.
Hmm, I'm not sure I'd advocate using a common/default name like "root" as the MySQL User: You can never be sure of when an exploit or script injection weakness will be discovered in the application software you're running, so I'd prefer to make the user name something a little more obscure (and the password very strong). After all, it's only the local host that should ever need to use these credentials and there's no point making life any easier for the mischief makers.

Re: Getting OpenSim on the Internet-Grid

Posted: Sun Mar 21, 2010 6:14 pm
by Nalates
I agree. Eventually I plan to move the info to the wiki. I will add a footnote on using 'root' verses something else.

Writing documentation on Alpha software is frustrating as it is changing so fast. I stuck with 'root' because of the tutorial/'how to' I was linking with. Once one is up and running then security becomes an issue. It was just too much for an opener.

Getting a SIM up and running for the novice has so many 'if ... then ...' situations everyone leaves different 'if' scenarios out. The more I work with the software I realize how much experience is involved in its writing. If one has a running SIM, the only change to get it on the Internet is one IP address in the configuration. Adding the SIM to a grid has a few more changes. It is all very simple if one understands how the SIM's work, which of course the new don't.

Re: Getting OpenSim on the Internet-Grid

Posted: Sun Apr 04, 2010 8:14 pm
by Nalates
Opening up OpenSim
As I learn more I find the basic install has several of the nicer features in OpenSim turned off. One of those is the OpenSim Scripting Language (OSSL). There are a number of new commands available in OSSL that are not available in LSL (Linden Scripting Language). When you are enabling these new features of the language you’ll start to understand why Linden Lab may not offer these in SL.

One of the handiest commands, so far, is a teleport command: osTepeportAgent(). This will teleport you upstairs or across the Hypergrid. A bug in the system currently limits how far across the grid one can travel. I understand the limit is 4096 grid units, not meters (4092x256m=1,048,576m). Grid units are the coordinates used to locate the grid. You saw that value in your region setup INI file.

It is felt the osTeleportAgent() is a high risk feature as it could be used to grief people by making a trick teleports. Whatever, the OpenSim developers built in security and disabled these features as the default setting in a basic install. On a standalone SIM one does not really need these features. Once you are on the grid they are more useful. So, here is how to enable them.

Enabling OSSL Extended Scripting Commands
Of course this is all done in the opensim.ini file. Make your backups of this file. There is quite a bit to add and some to change.

All the changes are to the XEngine scripting settings; [XEngine]. Fortunately there is an updated example on the OpenSim wiki. You will want to get that example and save it in your system as a text file, on Windows Notepad is a good choice to use as the paste recipient for text files. Using Word, Wordpad, or Windows Write for pastes from a web page can leave one dealing with some of the Rich Text features captured in the copy/paste operation. Notepad avoids that problem.

Even in the latest opensim.ini version the XEngine settings seem to be in two places within the file. One can correct that minor problem in their copy of it. The example XEngine changes bring all the XEngine settings together. One just needs to avoid having duplicates keys in their INI files. Can’t find any information on how OpenSim handles them.

First open your copy of the opensim.ini that you plan to change, not your only backup :/ Find this section of code:

Code: Select all

    ; ##
    ; ## SCRIPT ENGINE
    ; ##

    DefaultScriptEngine = "XEngine"
This example is about XEngine. I have not researched what other engines are available. I understand those running in Linux use Mono. This setting comes early in the INI file. I’ve left mine there. Just make sure you have the code in your INI. It should be there as it is part of the default INI included in the download.

Next find this section of the INI file: [XEngine]

This section defines how the engine works. This section can be replaced by the example XEngine section. It will need some changes after insertion to enable the features we want. But it should work without change. Only replace the XEgine section. There are other sections following it.

I’ll use the High Risk section to show how to change the new keys added by the example INI file.

Code: Select all

 ; **** THREAT LEVEL HIGH Block ****
 Allow_osTeleportAgent = true 
 Allow_osMakeNotecard = true 
 Allow_osMatchString = false 
 Allow_osTerrainSetHeight = false 
 Allow_osRegionRestart = false 
 Allow_osGetAgentIP = false 
 Allow_osSetRegionWaterHeight = false 
 Allow_osGetScriptEngineName = false 
 Allow_osGetRegionMapTexture = false ; osGetRegionMapTexture(string regionName)
 Allow_osCauseDamage = false ; osCauseDamage(string avatar, double damage)
 Allow_osCauseHealing = false ; osCauseHealing(string avatar, double healing)
 
The example file opens up several security risks. One can reduce the risk by using one’s UUID in place of true/false.

Code: Select all

 Allow_osSetRegionSunSettings =  253c47c9-9556-____________a893 
This restricts the commands to scripts created/owned by the you (UUID). Probably all the high risk commands should be false or your UUID.

Once you have the script changed get a copy of Whitestar’s script using osTeleportAgent(). You can use it to test if your changes are working.


Understand group ownership of a SIM affects these permissions. Rather than an individual’s UUID you may need to use a group UUID. I have not tested that. But, the group UUID opens up some risks. So, use caution and think it through.

Summay
This is all one needs to do to enable the extended commands. Some of them are way handy.

To find out how to use the commands Google on the command name and opensim. And... see OSSL Implimented