Maps in Github - Getting Started and Questions

classic Classic list List threaded Threaded
34 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Maps in Github - Getting Started and Questions

aaalafayette
Administrator
This post was updated on .
Maps in Github are arriving next release. Instead of storing maps in SVN: http://sourceforge.net/projects/tripleamaps/files/, we will be storing them in github: http://github.com/triplea-maps

From a game player perspective, a clean re-install of the game may be required for the next release (save game files will still work on the next version, so keep those).

Map makers, if they are submitting zip files to the developers, nothing will change for them. Otherwise, capabilities are being added that allows anyone to create maps, upload, and update maps all on their own. Finally, this simplifies life a good deal for the game developers, so turn-around times should be faster. Updated maps will go live in minutes/hours/a day, not weeks.

To get started on the new map making process, please dive in here:http://github.com/triplea-maps/Project, and here: http://github.com/triplea-maps/Project/wiki/Map-Makers:--Getting-Started-and-Workflow

** Update **
Github has launched a new feature that allows you to upload files using your web browser. With the capability to also edit files in place, you're pretty much now good to go with just your web browser. Here are the two links on how to edit/upload files:
http://github.com/blog/2105-upload-files-to-your-repositories
http://help.github.com/articles/editing-files-in-your-repository


Github map project issue queue: http://github.com/triplea-maps/Project/issues
* Can be used to raise questions that are global to all maps: https://github.com/triplea-maps/Project/issues/new, or to track tasks. Bugs impacting a majority of maps may be tracked here as well.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

Frostion
I have looked several times at your instructions, GitHub and all the guides you have posted there, and I must say that administrating map files this way does not seem to favor the casual non-programmer user/mapmaker. GitHub might be handy and smart if you got skills already, but Github it is clearly not made easy to for idiots to use. I have trouble just figuring out how to do basic stuff in GitHub, like releasing a totally new map or uploading files. For example, am I misunderstanding something, or does one actually have to install a Github program on the computer to release a new map?

Anyway, I would guess that new mapmakers will probably not jump in, learn Github and release or update their maps that way (unless they are already experienced with this type of website/program), especially if a GitHub-program is needed to release maps or replace files. But I understand that there will also always  be the option to “Post here to request having your map on the Map Repository uploaded or updated”.

Here are two  maybe unrelated questions:

Will there or are there some new rules or standards for map files, map names etc.? I mean all the issues with underscores, spaces, capital letters and so. Are there some new guidelines that need tos/should be followed?
The only thing in the “PRECONDITIONS FOR MAP UPLOAD” seems to be…
Also, the space character:
Instead of space use underscore (_) character.
Wrong: Example map.zip
Correct: Example_map.zip


Also, at least some maps in the GitHub triplea-maps dir seems to be out of date compared to sneakingcowards list here in the “ready for upload” post in this thread? The maps in GitHub triplea-maps seems to be about two months old, and sneakingcowards list is only 2-3 weeks old.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
This post was updated on .
Thanks for taking a look Frostion. I appreciate the feedback as well. I spent some quality time overhauling the docs, trying to simplify them down yet even more: http://github.com/triplea-maps/Project/wiki/Map-Makers:--Getting-Started-and-Workflow. Would you mind taking another look?


To respond directly to some of your questions comments:
Frostion wrote
I must say that administrating map files this way does not seem to favor the casual non-programmer user/mapmaker.
For the map admins they are set up so that if we can automate an action on one map, we'll always be able to run that same automation on them all. Otherwise, for map makers, they are set up to use the simplest Git work flow there is. Git makes the basic case of, "check out files from repository", "update them", "push them back to repository" to be quite easy. As long as nobody else is modifying the files, just one person, it's a pretty simple work flow. Thus, we've set up the repositories so generally only one person would ever really be modifying stuff.

With that said, there is a learning curve to Git. It's a very powerful tool that can support huge engineering teams, but it's also revolutionary in that it's the first of its kind to make the simple cases so  (relatively) easy.
 



Frostion wrote
I have trouble just figuring out how to do basic stuff in GitHub, like releasing a totally new map or uploading files. For example, am I misunderstanding something, or does one actually have to install a Github program on the computer to release a new map?
I hope the updated docs will help with this. You do need a local "Git" client to be installed on your computer. The Git client will help you download files from the repository, and helps you upload them again. The Map zip is created automatically after uploading files to a map repository.

In contrast, before you would have needed an SVN client on your computer to do the same thing. In general the admins were taking care of that part for you.


Frostion wrote
Anyway, I would guess that new mapmakers will probably not jump in, learn Github and release or update their maps that way (unless they are already experienced with this type of website/program), especially if a GitHub-program is needed to release maps or replace files. But I understand that there will also always  be the option to “Post here to request having your map on the Map Repository uploaded or updated”.
You're probably right. To one extent though, if the map admins have to do everything, and if a map maker is again waiting weeks/months for turn-around on simple requests, it will be some incentive to try and learn to do these things on your own. Worst case, Github allows for a ton of automation for the map admins, so even if it s a matter of posting the zip file and having the admins take care of the repository, Github will still be of huge benefit.



Frostion wrote
Will there or are there some new rules or standards for map files, map names etc.? I mean all the issues with underscores, spaces, capital letters and so. Are there some new guidelines that need tos/should be followed?
The names of the repositories have been standardized. The actual name of the map, as presented to the game user, comes from a field in the XML. So the folder name, and map zip name need to be named all lower case, replace spaces with underscores. The actual map name, defined in XML, can be whatever you would like.

There was previously a *lot* of variation in naming of the map zips. This made finding repo's harder than it needed to me, and made automation more difficult.


Frostion wrote
The only thing in the “PRECONDITIONS FOR MAP UPLOAD” seems to be…
Also, the space character:
Instead of space use underscore (_) character.
Wrong: Example map.zip
Correct: Example_map.zip
Almost: Correct: example_map.zip  (note the leading 'E' is now lower case)


Frostion wrote
Also, at least some maps in the GitHub triplea-maps dir seems to be out of date compared to sneakingcowards list here in the “ready for upload” post in this thread? The maps in GitHub triplea-maps seems to be about two months old, and sneakingcowards list is only 2-3 weeks old.
This is expected. I wasn't getting much traction with the other game engine admins, and I was tired of repeatedly recreating all the maps. Now that the config to download from Github is in place, I need to repair image issues and re-upload maps for one last time. I was waiting for that config to be accepted before spending lots of time making sure all the maps were perfect.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
PS: the git client recommended is "git for windows". It makes interacting with Git relatively easy. No command line needed, all point and click.  (more details on the setup page:  http://github.com/triplea-maps/Project/wiki/Map-Makers:--Getting-Started-and-Workflow)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

Frostion
In reply to this post by aaalafayette
OK, Im working trying to learn how to update my map files at GitHub via my re-installed GitGUI. And I am starting with the Star Trek map.
I “cloned” the map from https://github.com/triplea-maps/star_trek_dilithium_war.git . It is now on my computer. I can see that the XML retrieved from GitHub is a little different looking than the one on my PC.
There are obviously no empty lines in the new XML, and the tabulation of each line seems to have moved.

1. Is this because of some kind of automated cleanup of the XML that GitHub does?
2. Is the XML functioning identical to the original XML?
3. What happens if I change something in the XML at GitHub? Will it ruin something?
4. If I commit/push an XML to GitHub, will it then automatically do this cleanup?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
If you open the file in wordpad, or any other text editor other than notepad, does it appear normal? For example, if you open the file in wordpad or say Sublime (I recommend sublime over notepad and wordpad, it's a simple editor and does not insert invisible characters - https://www.sublimetext.com/download)

First thing to try, add a file called ".gitattributes" to the "clone" and add this one line to that file:
* text=auto

Git should then automatically convert to and from windows-style line endings

Frostion wrote
1. Is this because of some kind of automated cleanup of the XML that GitHub does?
Yes, one time on upload. Stripped windows specific style line endings. These line endings are pretty much *only* needed for notepad to display the file character. Every other text editor in the world should not have any problem with these files, except that one. The ".gitattributes" solution might actually fix this problem as well.

Frostion wrote
2. Is the XML functioning identical to the original XML?
Yes, assuming it's all just because everything now *appears* to be on one line when viewed from notepad. The XML is structurally the same. Just no more magic notepad only line ending.

Frostion wrote
3. What happens if I change something in the XML at GitHub? Will it ruin something?
Short answer, no.
Nope, you'll get a new version each time. The version that will be available to the game engine is pinned to a specific one. Once a version has been deployed, it takes a lot of work to recreate it. So some version might not be as good as others. If you break something we can go back to earlier versions.

Frostion wrote
4. If I commit/push an XML to GitHub, will it then automatically do this cleanup?
No. With the .gitattributes file solution, the conversion of the cleanup should happen automatically for you. My preferred solution in general here would be  to use anything but notepad/wordpad. Obviously I'll need to be a bit flexible on this perhaps, and could be worth me dropping in a .gitattributes file everywhere if it'll help others (and if it works well for you, let me know what you end up doing please, or if you get stuck)

*edit* fixed up the post, sorry about the initial typo's and poor grammarz...etc.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

ZjelcoP
Does this also aply to Notepad++? Like it, plenty of good options.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
Turns out the "issue" was formatting had been applied to the XML and wasn't line endings at all. Line endings only become a problem with the three Microsoft text editors: notepad, wordpad, and word (which all silently change them). So avoiding those three editors would be good, but even then still not the end of the world. Notepad++ is a very solid text editor.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

Cernel
In reply to this post by aaalafayette
Sublime Text may be downloaded and evaluated for free, however a license must be purchased for continued use. There is no enforced time limit for the evaluation.

Does this mean that it is illegal to actually use it for any projects at all (beside evaluating), unless you buy it?
History plays dice
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
Perhaps things are changing with sublime, used to be registration was free, but they also say this too: "There is currently no enforced time limit for the evaluation."

Atom.io is another popular editor. You can use notepad if you really like, but it's a bit annoying since silently adds extra characters to the text file (but it's not a big deal). To be clear, the difference in XML was formatting, not related to arcane line ending problems which are effectively only introduced when using notepad/wordpad (but even then, it's not that big of a deal)

 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
Best yet though, if you're editing an existing file, you can do it directly in the Github Web UI.

For example, to change the World at War XML, head to: http://github.com/triplea-maps/world_at_war/blob/master/map/games/World_At_War.xml,  then click the pencil icon, and github will let you edit the file place. If you are not an owner for the map, github will automatically create a fork for you which you can edit, and then request be merged to the official version.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
@All - some exciting news from Github, they just enabled drag and drop file uploads. What this means is that you don't need to know *any* git now to modify a repository. Drag and drop for new files/replacements, and then there is an on-screen file editor you can also use. I'll still need to update the docs, but wanted to let folks know about this new feature.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

sneakingcoward
In reply to this post by aaalafayette
hello aaalafayette,

is there something changed with map review and further upload ?
my person "sneakingcoward" was having a prelook at the zip file, before redrum was uploading it to the repository.

for this also at the forum "Preconditions for map upload" was a pinned topic.

1. has the zip file still be tested before upload ?
then please restore the above pinned topic.

2. is my work contribution still appreciated ?

3. you should pin your forumpost  "Maps in Github - Getting Started and Questions"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
sneakingcoward wrote
is there something changed with map review and further upload ?
yes and no, for perhaps another week or two the maps in SVN will still be "active". Then we'll start using the maps hosted in Github exclusively. We should have release fairly soon, a few weeks where in-game downloads will come from github rather than sourceforge.

There has been a bit of change. Before when a file was uploaded, it was effectively always made available for in-game download. Now maps can be uploaded at any stage, and there is encouragement to have them be uploaded earlier. But, to have the map actually be included as part of in-game download, then yes, it needs to be tested and likely fulfill most of the "Preconditions for map upload"


sneakingcoward wrote
1. has the zip file still be tested before upload ?
No, the upload repository is now a much more useful tool for collaborative editing. You could very well have something uploaded so multiple people could start working on it, or to take advantage of the automated map zip capacity.


sneakingcoward wrote
2. is my work contribution still appreciated ?
For sure. Is there anything to make you think otherwise? One goal of github maps is to make the map making process easier. We have tools for the admins now where it takes just a few minutes to do a map upload, rather than 10-30 minutes. So as admins we'll be able to do a lot more, you shouldn't see the multi-month delay bottlenecks anymore while waiting for your map to be approved. So the contributions are more appreciated than ever really.



sneakingcoward wrote
3. you should pin your forumpost  "Maps in Github - Getting Started and Questions"
Now pinned. I will admit there is some missing documentation here, not everything has been updated etc.. There is a lot to do. WE could use help with finding out-dated documentation and updating it.


3. you should pin your forumpost  "Maps in Github - Getting Started and Questions"
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

sneakingcoward
" So as admins we'll be able to do a lot more, you shouldn't see the multi-month delay bottlenecks anymore while waiting for your map to be approved. So the contributions are more appreciated than ever really. "

when i was helping with map review, in 2-3 weeks each map was uploaded, depends only when the upload was done, not the review.

this github looks maybe nice for admins, but will add an additional barrier for first time map makers, which are not releasing on a frequent basis.
to fork and push and....lets be honest, the people want to play and not to be involved to much in admin or software issues.
but we will see how it develops...



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
I reviewed the thread we had of "post here to upload your maps." Based on what I saw in that thread, half the time maps were uploaded relatively quickly, the other half of the time there were delays. On occasion requests for map upload were completely missed or forgotten. It's an example where forum threads are not good for task tracking. They're decent for conversations, but for keeping track of a dozen tasks that run for a few weeks/months, it's not so good. It's very important to be able to close issues so they can be marked as done.

So, the 2-3 weeks for upload is about right on average. But, that involved a lot of work from an admin.

Let's consider the old process:
- Map maker - post zip to a source forge thread. They are  done.
- Admin would download the zip, do an SVN update, replace the old map, do a SVN check-in
- Admin would then by hand update the bot servers

Today:
- Map maker posts zip to a source forge thread, they are done.
- Admin downloads the zip, runs it against a script, and they are done.

Given this, I disagree that we are adding an additional barrier. There are additional barriers a person can cross, but we are able to do things today which we could not do previously.

Consider the example of updating images:
Before:
- Find the one map thread for the map. Discuss in there that you want to change the map images. A person posts the sample images, and there is a lot of discussion.
- Next, agreement on the images. Everyone working on the map needs to then send the latest to the admin so that work can get pushed in. Then whoever wants to update the maps would download the latest zip, and apply their work. This would then go back to the admin for upload again. Meanwhile, in this process, only one person can work on any map at a time. Also, note this is just for one map.

Today:
- Someone wants to update images, they fork a repo, add their images, click the pull request button. The map owners then can click another button to approve the changes. It's mostly automated here and many people can all work on the same map now. We also have the ability to do bulk updates across all maps, which previously would have been just a nightmare to do.

I would suggest that we're actually removing barriers. Consider the NWO map thread where Cernel has wanted to update the unit images for perhaps half a year now. Since there is no clear owner on the thread, and because it's so difficult to coordinate it so only one person works on a zip at a time, it does not seem like that work will get anywhere. That's really a shame since it's all due to overhead, heavy process that does not allow for many people to collaboratively work on the same map. Then in addition because everything is being discussed in just one forum thread, it's really difficult to jump in on that, which blocks out new map makers. Even for those involved, it's really difficult to even keep the conversation straight.

My bottom line: we have not changed the process for map makers with Github, but we have allowed for additional, optional processes that can let map makers run independently without the admins becoming bottlenecks. The admins used to do ALL of the version  control work. Today they still will, unless a map maker wants to run ahead and do things on their own. There are some who are doing that today, it is quite easy to update items in Github now that it can all be done through the Web UI.

So  map makers are now enabled to do as much or as little of the version control work as they are comfortable doing. At the end of the day, whether it is SVN or Git, someone is doing the version control work. I think part of why this seems so radical is that we exposing this part of the process, now we allow map makers direct access to the version control. Before that was strictly not allowed since control of one map gave control over all of them (in sourceforge). So admins just quietly did *all* of the version control work, and it took significant time and effort to do that, and was a lot of  manual work. That's why map uploads took weeks, sometimes months.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

Frostion
It is surly good that first time uploads to GitHub can go though this forum and though an admin review/quality control.

But, it would also be a nice thing, for noob-mapmakers, if the door was open for the mapmaker to post update requests of their xml or similar. I think that maybe a required GitHub learning/usage could be the “additional barrier for first time map makers” that I could imagine.

I am sure that a mapmaker could easily learn how to update their own map via GitHub now that there is a simple drag and drop feature on the website. It is of course a shame that some of the good guides you, aaalafayette, made are now outdated, but now a guide could be even simpler and easier to understand and learn.

Regarding the time delay when posting maps requests here, I think that it could just be posted clearly that updates via the forum is a slow process, unlike mapmakers own updates via GitHub, and this would probably just motivate the mapmaker a bit to make a GitHub account, get ownership and drag drop to update.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

Frostion
In reply to this post by aaalafayette
How do I rename folder in GitHub website?

triplea-maps/star_trek_dilithium_war/BaseTiles
to
triplea-maps/star_trek_dilithium_war/baseTiles

(BaseTiles to baseTiles)
I don’t know if this has any practical meaning, but I would like it done.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

aaalafayette
Administrator
It looks like the Github website will allow the rename of files, but not folders. It looks like you'll need to use Git for Windows to make that change. Alternatively delete the old folder and re-add everything into a new (renamed) folder.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Maps in Github - Getting Started and Questions

Frostion
@aaalafayette
It seems that the only way I can delete the wrongly named directory is by deleting everything inside the dir. And it also seems every file has to be deleted one at a time!? Like as deleting all the 576 baseTile PNGs one at a time?

With Git Gui I tried to “clone” the existing repository at triplea-maps/star_trek_dilithium_war and get it down onto my PC. Then on my PC rename the directory to baseTiles (with small b), rescan, stage, sign (as “v0.2.5”), commit and push. Git Gui seemed to take its time and upload everything again, but when I visit triplea-maps/star_trek_dilithium_war, I still see “BaseTiles”, also not every file has been replaced and updated with the “v0.2.5” note.

The only thing I can think of is deleting the entire triplea-maps/star_trek_dilithium_war repository and upload the map again in its new and updated form. The newest version can be found here:
https://drive.google.com/file/d/0B32S8ObEkqQuaUF1RmVFU2VaQ1k/view?usp=sharing

In GutHub I have the option/rights to “Delete this repository”. But what happens if I do this? I think it would be better if maybe you did it, just like a first time map upload. I guess you are also the handler of the special gitignore, travis, build.gradle files that seems to be in the map ripositories?

I wish that GitHub would eventually be more user friendly and have a better web interface with more options. Then I would not have to bug you with all this trouble just because a directory is wrongly named.
12
Loading...