Outerra Worlds Forums

Middle-Earth World => General Discussion => Topic started by: Morcrist on January 22, 2014, 04:38:00 am

Title: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on January 22, 2014, 04:38:00 am
I'm in the middle of a game project for a studio and all of my me-dem projects are on a backup hd because I ran out of space on my computer. So I can't render anything out. The old renders are nowhere near the finished state of the latest terrain, it's not worth using to be honest. I can have a look to see if I still have the original output files from the last render before it got converted to the Outerra native format. Other than that, folks will have to wait a couple of months- could be a little as a few weeks- before I can go again.
 Our "policy" for want of a better word, is to not give out the terrain at full res. We could give you a 10K terrain and you could always upsample it.
I'm gonna' respond here since I don't want to off-topic the intro thread.

Since Minecraft has a height limit of 256, an 8 bit heightmap would be sufficient for my needs. I have no idea how getting an 8 bit heightmap from the elevation data would work, but if I have to wait...I'll wait.

When you say '10K terrain' what exactly do you mean? What's the dimensions? As a reminder I'm shooting for at least 24000x24000. 48000x48000 would be better!

So...no 8 bit greyscale heightmaps lying around? Doesn't have to be latest version or anything.

Take care.
Title: Re: 8 bit Heightmap?
Post by: Redrobes on January 24, 2014, 03:38:00 pm
I thought middle earth was done in mine craft. I saw the Hobbiton and Bree on some forum somewhere. You can check out the galleries I am not sure we have an 8 bit version anywhere. All of the stuff we use is float based height. I am not sure that 256 levels would adequately encode a terrain height. If you had a mountain 25600 ft tall then your res would be 100ft per value.
Title: Re: 8 bit Heightmap?
Post by: monks on January 24, 2014, 08:53:46 pm
We've got a 40K map in 64 tiles I think. I can't guarantee that it's the latest terrain. The folder says "previous terrain" so I think it's the previous terrain we ran- but again I can't guarantee that.  It's generally too high to be realistic, but it is nice visually. You can have those but you'll have to reformat and resize them yourself.
 I might even have the latest one. What's the name of the folder in the Mainmap directory where tiles are output Robes?

monks
Title: Re: 8 bit Heightmap?
Post by: Redrobes on January 24, 2014, 09:13:10 pm
I dont think we have a greyscale bitmap of the heights only. I think we would need to write a script to generate them if we needed it. We have only got HF2 type. We have a black and white only mask of the land sea at 10K res which is a good start. And we have a black and white mask of the hilly ground at same res and we have a mountain mask as well so we have a two bit solution... Those are in the preprocess 10240 section of the main map files and the Tiles Level_0_10240 section. Not bad for Minecraft tho -maybe ???
Title: Re: 8 bit Heightmap?
Post by: monks on January 24, 2014, 09:36:28 pm
Looking at the scripts, the tiles are output in C:\MEDEM\Mainmap\CalcTiles\Level_0_5000

I've got the latest run in there in the OuteraHeight. dir Yep 64 5K hfz. If you want to knock up a script Robes, convert them to 8 bit or whatever he wants them in...or if you're really busy I could just open them in WM and spit them out. That's if my computer has the hd space for the processing. I'm down to my last 150 GB. Prob easier with WM...I'll try it.
What format Morcrist? World Machine has TGA, RAW, BMP....

monks
Title: Re: 8 bit Heightmap?
Post by: Morcrist on January 25, 2014, 02:31:50 am
The easiest thing for you guys. Scratch what I said about Global Mapper. My 2 week trial runs out Feb 3, but I just found out I apparently forgot to read about a hard limit on the number of exports I can do. Woopsie!

So yeah, whatever image format is easiest/most space efficient for you. I've got paintshop pro so I can convert to whatever I need. TGA, BMP, JPG even. Any will do.

You mentioned '40K', and '10k' again. What exactly do those mean in terms of resolution? I keep thinking in terms of pixels. Is that right? 40,000 x 40,000 pixels? Or is that kilometers? 40 km? I'm clueless! Heh.

Meanwhile a few days ago I actually opened up NW Middle-Earth (http://fanmodules.free.fr/Our%20Resources/Maps/MERP%20-%20Map%20-%20Northwestern%20Middle-Earth%20map%20set_All%20in%20One_Carte%20TOTALE%208000x5400.jpg) in paintshop pro and started a new image. It's about 8000x5000 pixels and represents an area around 1700x1200 miles. Uhh, that'd make it like 1 pixel ~= 360 meters I think since 90 pixels was 20 miles on the map. I plan on upscaling it at least 3x to 24000x15000 and hopefully 48000x30000.

So anyway I immediately started a new layer filled with black and began tracing the coast with a 32 (0=bottom, 255=max height in minecraft). After a few minutes of tracing the coast I stopped, zoomed out...rofl. I had only gotten to like the Sea of Lhun, starting from the top northwest part of the map. Tedious. I eventually finished the coast last night. It's just a very small taste of the work involved in doing something like an entire map from scratch, I'm sure, but I've got a good imagination! Super kudos to you guys once again. /salute

I plan on doing some mountains tonight starting with the peaks and then working my way down. I don't like what I've seen in some other minecraft versions of NW ME where the water is all at the same level. Like globally. I know in real life water doesn't make 1 meter (1 block) drops as it flows to the sea, but I'd like to model the fact that the height above sea level gradually goes up from the coasts towards the mountains. And consequently you've got bodies of water like whatever lake feeds the falls of Rauros at a much higher elevation than say, the Anduin. And rivers would drop a block at a time over their concourse as they head towards sea level.

Just gabbing at this point. I'd love to see what I could do/how much time it could save me in my effort to get something into minecraft by using some of your data, so whenever you're able I'd be much appreciative!

And speaking of data, what did YOU guys use for your source way back? From what I've read online you guys are using the same source as that link above? ICE's MERP module maps? Just curious.

Thanks in advance, and take care.
Title: Re: 8 bit Heightmap?
Post by: monks on January 25, 2014, 09:33:49 am
Oh I know about tedious....the contour map I created- in fact the very first map after stitching the ICE maps together- was drawn at 12.5K (12,500 x 12,500 pixels). The contours were drawn with a 1 pixel brush. Look at your computer screen, see 1 pixel? Imagine drawing with that in a sea of white, then stack 12 of your computer monitors on top and to the side of your screen...lol I worked on it literally 14 hours a day for 3 months solid without a break. Honestly don't know what possessed me to do that...it had its own problems. Because you got lost in it, I had to make a first pass with a larger red brush. I'd use this a guide in the 2nd pass. Also, it would be so easy to get gaps of the odd pixel here and there which you had to fix because you needed an unbroken line. If you painted too quickly this would happen so you could only go so fast. So I had to make a 3rd pass over the lines to check them. Not to mention actually working out the contours from the ICE height map (Guide to M-E I think?) and the height refs I compiled from the scenarios. Another thing, you have to make sure that your 1 pixel lines always connect with a face, rather than on a corner, otherwise they get treated like broken lines (can't remember -think it was the raster to vector process). You had to go through thee lines with a 1 pixel and dot the corrections- now that was super tedious. Luckily I had my intuos and set up the buttons to pan/zoom, etc. No way I could have done that with a mouse.
http://www.me-dem.me.uk/galleries/Map_images/ContourswithRivers.png (http://www.me-dem.me.uk/galleries/Map_images/ContourswithRivers.png)

 That's why you need contours- for the realistic water levels. I don't know of any better way of doing it. But it's a pita, and a bit of an obscure workflow- more GIS than anything else. It's like a table-top effect- like with model railways on a table tennis table. You can see essentially it's all flat with stuff plonked on it.
 For Minecraft though you could possibly get away with using greyscale heights and then digging out the rivers, but I still think contours is a more reliable way to create the rivers properly. The problem is if you use greyscales and then blur them together, blur is a uniform operation, and you don't necessarily want that since you end up closing off narrow river valleys. Contour interpolation on the other hand preserves the exact height at those points.
 There is a better way now as of last year. It took several phDs mind you to pull it off. Funnily enough, this is exactly how I said it needed to be done some years ago, including the polygonal watersheds and Strahler stream ordering. I wish I could program!
SIGGRAPH 2013 - Terrain Generation using Procedural Models based on Hydrology (http://www.youtube.com/watch?v=JCsj0v-wmIM#ws)

Really you need all of your data in a GIS vector format so you can render to planets with scale independence. Unfortunately .ai or .svg don't support per vertex z, so you can't use those. You can convert raster to vector with a little app called WinTopo which I use. You have to tag all of the vector contours with z values- another big and tedious job. At some point you also have to connect all broken discontinuous vector contours as well...because WinTopo algorithm tends to break up lines into smaller segments.
 Then you got to to create all of your rivers and roads.....ouch... ;D.., I'm so glad I'm looking back at it now cos I don't think I'd have the patience or sanity for it anymore.

I'll try outputting 40,000 x 40,000 (40k) in 64 5K tiles in 8 bit bmp format.

monks
Title: Re: 8 bit Heightmap?
Post by: Redrobes on January 25, 2014, 02:34:05 pm
If your able to do that Monks then that would be useful. The way I would have to do it is to render out all of the 1600 tiles then resample them to 350 pixel tiles then blend them all back into a 10K square PNG. It could be done but for one thing I dont have the latest height tiles tho I think mine are probably good enough. But its a lot of work to generate them.
If we put a lot of effort into it then it would be possible to generate the map and export the height and water tables out to 8 bit with the colour map for minecraft. We could do the volcanic and trees and water bodies as well. Potentially we could make a MeDem in minecraft but I haven't used that app yet. Just seen my young cousins playing around on it thats all. Personally whilst there isa free Outerra demo for MeDem which also fills in the sub 100m level with procedural fractal landscape and sorts out the 3D terrain generation then there is no point in looking around at any other terrain visualization tool.

I dont think Monks covered that last bit tho. We have a final map of 40K x 40K height points and colour pixels. Its generated off of an export of contour maps at a 10K res. We also input some more hand made or script made data at 10K for some parameters like forced vegetation and temperature maps etc so that the terrain generation tool gets the right sort of output terrain for the area being generated. We export for ViewingDale as a 2D set of maps, geomapper set of map layers for a "google maps" type thing that someone does for us and then also Outerra.

Anyways - I hope Monks can export something. I expect that you will discover that most of the terrain will consist of very low value heights and that the blocking will be quite large. But then that's what Minecraft looks like so maybe thats ok.

Title: Re: 8 bit Heightmap?
Post by: monks on January 26, 2014, 09:04:43 am
I'll try it here. I need to know though what format he wants first...

monks
Title: Re: 8 bit Heightmap?
Post by: Morcrist on January 26, 2014, 12:04:58 pm
Any image file format, actually. Whatever is easier? BMP would be fine.

I really appreciate you taking the time to do this for me. Can't wait to see what it looks like!

Thanks, and take care.
Title: Re: 8 bit Heightmap?
Post by: monks on January 26, 2014, 07:56:41 pm
40K heightmap @ 64 x 5000 px is up. You should have received a mail from GoogleDrive.
 It's bmp and you may have to flip the tiles and/or ordering scheme to get the correct orientation for you app. It all depends on the app you use. We tend to use Perl scripts here.
 Don't forget to credit us. We don't usually give away our data at full res. In this case 8 bit is fine. ;)
 Let us know how you get on with it.:)

monks
Title: Re: 8 bit Heightmap?
Post by: Morcrist on January 27, 2014, 04:45:14 am
OMG!

That is awesome. Monks, I can't thank you enough. I really, really appreciate that. It took me a while to figure out what I was looking at with it divided into 8x8 chunks, but once I did...sweet!

Only thing is, this work has really underscored the age of my hardware. With even 8 Gb of ram I keep running into Out of Memory errors using my (dated) Paintshop Pro 7.04. Heh. Things are slow too. My cpu is only a dual core Pentium (not even an I-xxx). I created a new 40000x5000 image to mosaic a row of your 5k chunks into and only realized PSP maxed out at 32767 width when I got to the last 2 chunks in the row. Derp.

I've been drooling over MSI's 128 Gb motherboard on newegg.com....

But yeah man. Super awesome! I may have to just create the chunks in minecraft using World Painter one chunk at a time and then use MC Edit to stitch the chunks together. Soon as I get something in-game I'll link a pic or something.

You're the greatest. And crediting you guys is the absolute LEAST I can do. Thank you, thank you, thank you!

Take care!

Title: Re: 8 bit Heightmap?
Post by: monks on January 27, 2014, 10:21:24 am
:)...that's great. Yes it's a beast. I did most of this thing with 8 GB RAM though. It was only right at the end (of this run) I upgraded to 32 GB.  So it should be possible. One more reason to use hfz workflow...compressed GIS format we got developed. bmp are not good. Can't you convert to lossless png? Hmm, no, 8 bit. Right.Is there no 8 bit compressed format? Robes will know.
 I guess you'll have the best Middle-Earth terrain in MineCraft now...well in terms of sourced real world terrain. I'll send you the rivers and roads as well if you can use them. I saw painting of rivers by hand. Is there no way to use an image overlay over the terrain so that you can use it as a reference?
 128 GB...? Crikey...filling up that board might be expensive!

I'm going to delete the terrain tiles later this eve, so make sure you have them all.

monks
Title: Re: 8 bit Heightmap?
Post by: Redrobes on January 27, 2014, 04:46:51 pm
PNG is lossless so for 8 bit should be fine. There is a 16bit PNG but implementation stability varies. Personally I think a PNG or BMP per tile would be fine but I dont think many apps would handle 40Kx40K images. Generally I believe paint shop pro would be ok up to about 16K then it gets problematic. Ideally you need a 64 bit app and run a 64bit version of windows. Even then tho the image file formats are not geared up for very large image sizes. TIFF is probably the best in that regard but TIFF has lots of issues which is why we dont use them. Its a good idea to stick with tiles and page them in as required.
Title: Re: 8 bit Heightmap?
Post by: Morcrist on January 28, 2014, 05:46:37 am
Is there no 8 bit compressed format? Robes will know.
Actually, the tiles you outputted were in 24 bit BMP format. They were 72 Mb each (5000x5000x3 bytes). I'm lowering them to 8 bit as I work on them so that's helping (24 Mb vs 72, or 1.5 Gb vs 4.6).

I guess you'll have the best Middle-Earth terrain in MineCraft now...well in terms of sourced real world terrain.
Heh, thanks to you guys' blood, sweat and tears. But I doubt I'll have anything remotely playable for quite some time. I've been playing around with the tile that has the Lonely Mountain and Withered Heath (x=5, y=3), which brings me to...

I'll send you the rivers and roads as well if you can use them.
How does that work, data wise? I mean, the tiles you sent me are 0->255 elevation maps, basically. What would a "river" tile be composed of? Or a "road" tile?

I saw painting of rivers by hand. Is there no way to use an image overlay over the terrain so that you can use it as a reference?
Oh sure. World Painter allows overlays, and I've used them to great effect in my tinkering.

But I'm wondering something at this point. I've imported the x=5, y=3 tile into World Painter as a heightmap. I can see where, for example, the river coming down from the Lonely Mountain flows down into the Long Lake and out the southern end of it. But it almost looks like the riverbed coming from the mountain was eroded, which is fine and natural looking, but the actual lake itself looks gouged out of the terrain at a uniform depth. As if it were done by hand. Likewise the large swamp to the west of the northern end of Long Lake. How did you guys actually do the rivers and lake beds, etc.? I'm thinking that since you weren't concerned with actual voxels you didn't concern yourself with what was UNDER the water. Heh. You just wanted something that would render correctly as a body of water, yes? It didn't matter how deep the lake was, or the swamp, etc. Just my guess.

Anyway, nothing I can't touch up in the BMP before importing.

There also seem to be some other eroded looking "riverbeds" in places that the topo Middle-Earth map doesn't have. I was curious about those as well. For example the long, thin riverbed that seems to originate east/southeast from the Lonely Mountain (almost halfway to the Iron Hills) and then flows south/southeast all the way down to where it finally meets up with the Celduin. Again, I can't find it on the topo and was just curious why it's there? :)

128 GB...? Crikey...filling up that board might be expensive!
Yar, but I could start with 32 Gb and work my way from there budget permitting. Heh heh. And there's the rub. It's not permitting right now, nor for the forsee-able future.

Thanks again! Take care.
Title: Re: 8 bit Heightmap?
Post by: monks on January 28, 2014, 09:08:14 am
Ah, there ya go, 8 bit.

We are concerned with underwater- bathymetry is all there and will be developed further, but Outerra doesn't support water at multiple height levels: no lakes. Long Lake is a very deep lake (I'm using ICE here), but it's made too deep by the introduction of the surrounding heightmap. Our next terrain will be much closer to where we want to go. It'll be lower overall, but still maintain a lot of the height contrast that's in the video terrain (not in the terrain you have, which is the release terrain- the video terrain was the version before that). Add to that the fact that we haven't really paid any attention to lakes yet anyway. So, it's your call.
 Rivers will be 8 bit bmp masks, not height data.
 I added a lot of rivers that weren't on the map. There was vast waterless regions.

monks
Title: Re: 8 bit Heightmap?
Post by: Morcrist on January 28, 2014, 12:21:34 pm
Ahh, gotcha'. I couldn't help but notice how high the plains were in x=5 y=3. Between the Lonely Mountain and the Iron Hills to the east the plains looked like around 100 or so high (out of 255). Clouds are hardcoded to 128 blocks high, so you're standing on the plains with clouds only 30 blocks (~100 feet) above you. The Lonely Mountain only went up to about 180. I realize with only 8 bits of height to work with things are going to be squashed together, and it's not a problem. In Minecraft, standing at the base of Erebor looking up it still looks like a mountain.

But I wonder what it would look like with better contrast between low lying and mountainous areas. How much effort would it be to export the same BMP tiles with the older "video" data set? Or do you think it wouldn't even be worth it? Plus I hate to think of you spending a large amount of time doing anything for me since I'm basically just mooching off of you guys. Heh. Not that I'm not supremely appreciative!

Anyway, any chance you could export the river and road masks for just x=5, y=3 and let me see if they would even be useful? If you can't export just one tile then don't worry about it. I can hand paint them in myself using a topo overlay.

And you adding rivers, that makes sense.

Take care!
Title: Re: 8 bit Heightmap?
Post by: D3Souz4 on January 28, 2014, 05:49:25 pm
I made an account and wanted to chime in because I worked on a ME map in Minecraft for over 2 years and ran into a lot of the same problems you're getting.  Unfortunately I started long before world painter was released so I had to use other mods to custom generate terrain.  Also my map was originally created when the height cap was only 128.

My original goal was to have rivers change height with the terrain as they were flowing out to sea but then when you went to a mountain it just looked like a small hill.  My solution at first was to have just very steep walls so mountains pretty much just blocked your path of travel.  I eventually re-did it so that there was much less of a gradual slope up and the mountains were a bit taller.  I still cut the tops off though.

When the height was increased I created some mountains from heightmaps in world painter and just dropped them in with MCEdit.  They looked like shit.  I eventually cut out sections with MCEdit, ported them to a new map, opened it into World Painter, custom made the mountains, and then ported the terrain back in.  They look much better now but they can be improved upon.  My map right now is 45k x 45k blocks.  I'll post some links below with old pictures.  Its gotten to the point where it is so large I'm not sure where to host the file and haven't taken any screenshots in a while.  I started the map in December of 2010 and I really got burnt out working on it.

Original Map: http://i.imgur.com/Hia2R.jpg (http://i.imgur.com/Hia2R.jpg)
Original Heightmap: http://imgur.com/xtW2C (http://imgur.com/xtW2C)
Early Screenshots: http://imgur.com/a/5JDF7#aqpM7 (http://imgur.com/a/5JDF7#aqpM7)
Updated Screenshots after height increase: http://imgur.com/a/vwRnw#12 (http://imgur.com/a/vwRnw#12)
More screens: http://imgur.com/a/OypFX#15 (http://imgur.com/a/OypFX#15)

I've since expanded the map. It used to on go as far east as the shire, and south to just past Minas Tirith.  I've now completed the entire coast down to the Gulf of the Anduin.  I do not have any pics on this computer (I'm at work) but I can probably get them when i get home.

edit: I should note that Orodruin and most of the rivers have been greatly improved since these screenshots.  I was just looking back at them and saw how low the water was and the river banks are so steep.  I raised low areas up so that the water table could be raised and fix those spots.  Orodruin was probably re-done 10 times from scratch before I got it looking decent.  Still really need to fix Mt. Mindolluin.

edit 2: Found more screenshots. 
All of these were prototypes.  Only the Black Gate looks remotely the same in my current version. http://imgur.com/a/zi6mW#0 (http://imgur.com/a/zi6mW#0) . Picture #2 was supposed to be Durthang.

More old ones: 50/50 of things kept: http://imgur.com/a/92mUQ#0 (http://imgur.com/a/92mUQ#0) (Moria shot in there and a Balrog skin)
http://imgur.com/a/WztBO#0 (http://imgur.com/a/WztBO#0)

Old map: http://imgur.com/a/qAz52 (http://imgur.com/a/qAz52)
Title: Re: 8 bit Heightmap?
Post by: monks on January 28, 2014, 08:14:52 pm
 Unfortunately I don't have the older video data set. You can hang around and wait for the next release if you want but I'm not making any promises on when that will be. Obviously it'll be as soon as practically possible. I'm working on a small studio project and it's maxed out my hard drives. So I've got to really get that to a point where I can make space.

Seriously guys, you 2 should team up or something. It makes no sense for 2 people to be duplicating/repeating effort...well, I understand it's also a personal journey bringing it to life, but projects of this scale need teams of people.
 Maybe consider contributing your skills here?

 Is this the M-E I've seen on MineCraft?..pretty awesome Isengard there. Yes, looking at the other pics. Awesome rendition of Middle Earth!  8)
 We've got a ways to go yet until we have it that interesting frankly, there's very little in our world as of yet, but this is a long haul project. It's become a bit of a proof of concept for Outerra since Outerra is middleware, it shows the possibilities of creating planet sized games- although our's is nowhere near that size of course. But having such draw distances does throw up new technical problems, but also problems for terrain artists. Our first draft was using purely procedural terrain and it looked awful from orbit. For a few years I'd intended to move over to real terrain, so that was the catalyst. Mordor was an exploration of that workflow.
 Another advantage Outerra has is extensive use of the gpu. Cameni does as much as possible on the gpu. That opens up the potential to even have new terrain modelling tools on a planetary scale. There are technical papers out there which could solve this whole conundrum of correct hydrology which will be necessary for artist engineered worlds like this (see previous post). I think this is where games are headed.
 In addition Outerra is a hybrid engine which is the obvious choice in my opinion. It can handle heightmaps and procedural terrain generation in sync. there are a few procedural planet generators, even galactic scale stuff but you can bet every planet will be pretty much in one fractal flavour.
 Minecraft looks wicked but I've thrown my hat in with Outerra. The best part of Minecraft must be the possibilities with volumetric tunnels. I think it's a platform more for builders than players if you see what I mean, hence the name. I'd like to see Outerra's support of real world speliological data format or something for creating cave systems on a grand scale. Might not be necessary, but it'll have to be GIS complaint and a vector interface; it'd have to be shp since ai or svg don't support per vertex height. Draw your tunnels as line vectors. Outerra procedurally fleshes them out. The rest can be created by more tradtional modelling methods. Then there's voluemtric erosion to create the additonal details you haven't got time to hand model. Robes briefly looked into this way back but that before gpus and OpenCL and the like. That's technically doable now.
 Since Outerra handles GIS, it's also got access to real world data and file formats, etc. To illustrate the advantages of that, the YT vid I posted, they use Strahler stream order which is used in the GIS industry to organise stream networks.
 
 You can always hook up with us, but I suspect you've got a lot vested in your goals and projects. Maybe you've outgrown MineCraft D3Souz4, I don't know.

 We're looking for someone to head up the 3D modelling, site design, all that stuff. Skill is less important than enthusiasm and perseverence.

monks
Title: Re: 8 bit Heightmap?
Post by: D3Souz4 on January 28, 2014, 09:15:24 pm
I would certainly love to help but life gets in the way.  Its partly why I stopped working on the Minecraft map.  I haven't touched it in nearly a year.  I would definitely be willing to share all resources I used to anyone hoping to make a map.  Turning my map over, which I would love to share, defeats the purpose though.  Its really about the journey and the joy of making a world more than the finished product.

edit:  I just saw your list of things you could need help with.  I'm not sure about 3D Modeling since I haven't done it in about 7 years but I have done the research and have the experience to know proper sizes.  I bought and read nearly all the books pertaining to Middle Earth and reread them when building my map.  If you have any questions or want to discuss any landmark I'd be happy to help.

Also what program do you prefer for modeling?  I have Bender on my computer but I never really learned it.  I used Unigrafix when i worked as a Drafter and ProEngineer in High School.
Title: Re: 8 bit Heightmap?
Post by: Morcrist on January 29, 2014, 02:39:36 am
D3Souz4, hey man. Really nice work. Very impressive. I'd love to try and import some of your structures into a map once I get monks/robes heightmaps ran through World Painter. It's really exciting to me to be able to try and create a map of NW ME based off of a real-world modeled elevation map. To be able to have awesome structures like the ones you created would just be icing on the cake.

I was working with thebearman85's massive middle-earth map (http://www.minecraftforum.net/topic/1691501-wip-massive-middle-earth-map-with-sample-downloads/) for starters, and trying to import various mappers' works into it. I'm seriously anal about a lot of things in life, especially Tolkien, which led me to being a little dissatisfied with a couple of things he was doing (not that his work thus far isn't awesome!) But that led me to coming up with the bright idea of begging the ME-DEM guys for something I could use that might be a little more realistic. :P

My main problem with importing structures was the fact that the mappers used flat world to showcase their structure (i.e., using almost the entire 0-255 height range). This ended up in me not being able to fit the structure into a real world modeled map without scaling it down. Take Divici's Minas Tirith (http://www.planetminecraft.com/project/minas-tirith-11-scale-1818179/) over on Planetminecraft. Epic, right? Well, when I scaled it down to 50-75% I took a huge fidelity hit. Parts of stairs were missing, walls were gone, etc., etc. Really frustrating. Same with Isengard.

It looks like the structures in your map, however, are built into the existing topography. This would probably work out great.

Anyway yeah, really exciting to see somebody else that has worked on something like this! It looks like you were coming more from a movie angle than the books? I plan on trying to do my own Minas Tirith based on the books and some paintings by Nasmith. And Moria. I want my Moria to be as close to the books as possible. I have a serious pet peeve about how PJ portrayed Moria in the movies. Makes me angry just thinking about it. Bleh.

Gonna' try and fiddle with those BMPs some more. I really want to get a methodology working that produces nice, believable rivers that run downhill. :)

Take care.
Title: Re: 8 bit Heightmap?
Post by: D3Souz4 on January 29, 2014, 04:05:07 pm
I've never seen that "massive" Middle Earth map before.  Funny thing is mine is almost 4x larger and includes less of the map.  I think his does look better from a cartography view though.

The model sizes are closer to normal because most of them were built before the height was increased to 255.  The exception being Barad-dur.  That was remade so that it could be more accurate and is 254 blocks tall.  Minas Tirith actually didn't most of the tower on the top level due to the previous height cap so that is more like 155 blocks tall now.  I tried to stay with a book perspective but many of my models were donated to save me time which gives it a movie look.  I thought Barad-dur was done excellently in the movie though.  Lothlorien I used a forest generator to get massive trees and just made tree house likes steps and platforms through them.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on January 30, 2014, 06:05:25 am
Figured out a way to get much better contrast between mountains and plains in the BMP before importing into World Painter. Paintshop Pro has a Histogram function called 'stretch'. It preserves the highs while darkening the lows. This serves to keep the high, majestic mountains while at the same time lowering the plains. It looked much nicer.

As for rivers, I tried painting water below a certain threshhold, and "walking" it up the slope towards the source one or a few blocks at a time. This looked ok, but not great. Next I'm going to try a lot more prep and actually make the river a couple blocks deep everywhere it's supposed to be, then use the fill command judiciously and see what happens.

Monks, I've noticed odd striations in the x=3 y=4 tile's mountains (which covers Eregion, Moria's west and east gate, Lorien, Fangorn, all the way down to Helm's Deep). Notably the Misty Mountains around Moria and northwards. Why are there so many valleys in the actual range? Is that normal?

Just wondering.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on January 30, 2014, 09:13:31 am
That's one way of doing it. Photoshop has the levels and curves functions- pretty much allows for the same kind of manipulation. That's what I use to alter the satellite dems.
 Yes, I think you'll find that our dem is far from perfect. Heights are out as mentioned. Also, rivers might well require a lot of work. We use a bottom layer to create a gradient along them but it's got its problems. I need to put a lot of work into them, but I'm still finishing off the broad strokes of the terrain, and frankly I could even continue with that for some time if I wanted everything to be just so, but that's not really practical.
 I'll take a look in Outerra, see what gives.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on January 31, 2014, 12:06:31 am
Wow there is a lot to take in on this thread. I have seen Minecraft had some Tolkien stuff but not really gone into it to find out much. The low res of Minecraft is a serious impediment to making a good map but on the other hand its a huge benefit to making buildings. Being just so ridiculously easy to build things is such a low barrier to entry for anyone to get involved. I think thats our problem with the requirement of Outerra in the 3D department. It needs Collada format files but sketchup and OBJ could be converted no probs. Blender would be fine I am not sure about the other one mentioned as I had not come across it before.

Regarding scaling of the height in Minecraft, what kind of sampling did you use. It sounds like you used nearest neighbor or pixel sampling in which case you would loose a lot of detail. Tho you must loose some, if you used a more intelligent sampling algorithm like say Lanczos then you would probably not have noticed it quite so much. But fundamentally having only 255 height values is a problem.

Im gonna have to chew on these posts a bit more. Interesting stuff - thanks for posting them. I am wondering if it is at all possible to get the minecraft data and then strip off the buildings and see if they can be of any use whatsoever. I don't think so but its a thought just rumbling around my heat at the mo. I really like some of the interpretations esp Barad Dur.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on January 31, 2014, 03:27:50 am
My apologies for being lazy, monks. Here are the relevant tiles (in jpg format to save space) so you can see 1) what I'm talking about re: the "striations", and 2) the before and after using PSP's stretch function.

Before and after...

(http://morcrist.com/images/minecraft/Middle_Earth_DEM_Project_x3_y4 ORIGINAL_thumb.jpg) (http://morcrist.com/images/minecraft/Middle_Earth_DEM_Project_x3_y4 ORIGINAL.jpg)        (http://morcrist.com/images/minecraft/Middle_Earth_DEM_Project_x3_y4_thumb.jpg) (http://morcrist.com/images/minecraft/Middle_Earth_DEM_Project_x3_y4.jpg)

I really like the results, speaking from in-game of course. As I said, the image on the left produced plains west of the Misty Mountains at a height around 100 blocks (out of 255) with the cloud level only 28 blocks higher. The image on the right produced plains at around 50 blocks. Doesn't sound like much but it makes a huge visual difference in game. :D

Also note the striations in the Misty Mountain range. Notably the long, uninterrupted valley extending from the top of the tile all the way down to Isengard. In addition the spurs almost seem like they're uniform in angle. To a large degree at least, they appear to mostly extend from the ridges in the same directions. Contrast that to this (http://eol.jsc.nasa.gov/sseop/images/EFS/highres/STS058/STS058-89-32.JPG) picture of a RL mountain range in Colorado, where the spurs seem more random in formation. Or do they? lol maybe it's just me. Feedback appreciated.

For the record, in game it looks nice. I'm not gonna' lie. Just something bothers me and it may just be me.

Here are some in-game screens of the area where the west gate of Moria should be. I've added some trees and a door to locate it...

(http://morcrist.com/images/minecraft/screen001_thumb.jpg) (http://morcrist.com/images/minecraft/screen001.jpg) (http://morcrist.com/images/minecraft/screen002_thumb.jpg) (http://morcrist.com/images/minecraft/screen002.jpg) (http://morcrist.com/images/minecraft/minimap1_thumb.jpg) (http://morcrist.com/images/minecraft/minimap1.jpg)

The last is a pic of the minimap of the area. Sorry I haven't explored all of it yet. It shows where the west gate would be, and the east gate which would be a little to the west of the lake on the right (Kheled Zharam?). Just noticed it has some artifacts on the west end. I just imported part of this one DEM tile, so that's where Minecraft started auto-generating extra terrain.

The 2nd screenshot shows a bit of the scale standing at gate level (height around 70) looking at the peak above (height around 180). The Sirannon (river flowing down from the west gate) was just a placeholder job. I'm going to set my map in the 3rd age, around the time of the Fellowship for continuity-sake.

Of course the dirt/mossy stone/stone/snow on the mountains leaves a lot to be desired. I just picked some sensible height values and made it use those blocks if the map at a given spot was >= that height. I'd like to eventually at least attempt something in the vein of this (http://www.youtube.com/embed/ItYcoZcis7E?start=500) gentleman's fine work as far as detailing the mountains. This guy is just awesome. Note that he's using a custom texture pack that gives more variations in the stone blocks, but still! Sweet!

Enough for now. Laters!

EDIT: Forgot to answer your questions. World Painter takes the BMP, which is an 8 bit (greyscale) heightmap, and imports it straight into a minecraft format. If in the BMP a pixel = 0 (solid black), then the block at that x/y gets placed at height=0. If the pixel = 255 (solid white) then the block at that spot gets placed at height=255. So there's no sampling per se. What determines WHAT block gets placed at that height is up to you on import. As I mentioned I chose a handful of parameters that basically takes any height less than say 100 and makes it grass, anything from say 100-128 would be dirt, 129-150 would become mossy stone, etc.

But yeah, having only 256 possible heights results in some low fidelity landscape. Quite fun and playable though!

As far as cutting out some buildings from minecraft maps...yes indeed. That's exactly what I was doing prior to getting DEM data from you guys. I was taking mapmaker's works that they've made publicly available (http://planetminecraft.com (http://planetminecraft.com) for example), opening them up in MC Edit (another Minecraft world editing tool - think less paint, more CAD) and cut and pasting structures out of them. I would then import the structure into the part of the ME map I wanted to put it in. All super easy to do stuff, just time consuming especially on low end hardware.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on January 31, 2014, 09:15:59 am
 Looking good man :)
The striations...not sure what you mean but I think you're referring to the valleys; to the terrain itself rather than any artifacts. Yes, they are a bit streeeetched I think but I can live with it for now. If you want to make any changes feel free. It's a wip. You'll find lots of imperfections in it. For eg, some of the mountains on the north-west flank of Mordor are too tall and thin to be credible spires, but then in Minecraft they might look suitably caricatured. The spurs yes, I think they're a bit too crisp to be natural, but they look ok in game and are probably what Tolkien had in mind. But then, if he had our tech, I think he'd make them look more natural.
 What you did with the heights is essentially what we'll do. We have a base layer. It's way too high at the moment. What happened is, way back- we decided that the base layer would hold the entire span of the dem data but in low detail form. All terrain data would be replaced. Then I decided to use the base layer to add to, which was way better. The base layer remained at the original span for so long it became part of the furniture, and in our biomes tests we were a bit foggy about just what the problem was. Then I saw the elephant in the room. Next run will change that.

I think we should get those buildings if we can from Minecraft. All we need right now is very basic placeholder forms for the locations- that's what we were planning to do anyway. Any help would be appreciated. ;)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on January 31, 2014, 03:55:13 pm
monks, Redrobes, what are your thoughts on me sharing those greyscale tiles you exported if anyone asks? Nobody has, as of yet, but I'd like to know how you'd feel about me uploading them to my website for anyone to download.

Completely up to you.

Thanks, and take care.

EDIT: Oh, and as far as exporting some buildings from Minecraft maps...how would that be useful for you guys? I mean, it's in a proprietary format as far as I know (.schematic (http://minecraft.gamepedia.com/Schematic_file_format)). How would you go from Minecraft blocks to a 3D model? I can't find anything after a quick search, but I dunno'.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on January 31, 2014, 04:17:13 pm
How about setting up a terrain for models deal here? At least it will give us the option of using the MineCraft sites as initial placeholders. Who has a repository of site MineCraft M-E models?
 Other than that, I don't mind as long as the original source of the terrains is cited... but this is a democracy so Robes has to cast his vote too.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on January 31, 2014, 05:13:03 pm
Guess what?! Found something to export minecraft maps to .obj files. And it worked! I created a ~140 MB obj of Divici's Minas Tirith. I'm uploading it to my website now. Will provide a link when it's done in case you guys can do anything with it. Note that I'm not uploading the textures right now, just the obj.

Although I've been downloading obj files viewers with no luck so far. On my third one. First one crashed. Second one didn't do anything. Third one got to 52% then hung. Wtf.

Does anyone know of a solid (free) obj viewer that can handle 140+ MB file sizes?

I'm really excited to see what this would look like in Outerra.

EDIT: Found an old post from Redrobes suggesting MeshLab. That did the trick.

EDIT: I'm an idiot. Re-uploading as a ZIP file. Derp. Cuts size from like 140 MB down to 20, and WITH the textures. I was checking it out in MeshLab...it modeled every little room and everything. Would be really cool to see this in Outerra and walk around in it possibly, if it could be scaled correctly.

EDIT: http://morcrist.com/files/minecraft/MinasTirith.zip (http://morcrist.com/files/minecraft/MinasTirith.zip)
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on January 31, 2014, 05:25:14 pm
Yes, try MeshLab its very good at handling large models. Also, if these files are so big then its likely that they are objects made up of loads of cubes which is not ideal. However you can just take the overall point cloud from all the cubes and mesh out a skin over it. See if you can zip an OBJ file on the forum and lets see what it looks like. Blender will also load in obj files no probs. Theres also 3DExplorer and Wings3D will also do it.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on January 31, 2014, 07:57:16 pm
Awesome!  :) I've got 3DS Max here so can always open it in that as well. Is it available for download?

//Nice one downloading now!

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on January 31, 2014, 08:31:04 pm
Without the textures in Max...

http://forum.outerra.com/index.php?topic=843.90 (http://forum.outerra.com/index.php?topic=843.90)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 01, 2014, 04:30:09 am
Helm's Deep (http://morcrist.com/files/minecraft/helm's_deep.zip). It's like 10x smaller than Minas Tirith (2 Mb). EpicQuestz (http://www.planetminecraft.com/project/helms-deep-1927706/) over on http://planetminecraft.com (http://planetminecraft.com) did this one. I really liked his Minas Tirith, but I preferred Divici's. Close call though.

Redrobes, Minas Tirith was just huge. Heh. But yeah, you're right...it's modeling every single cube no doubt. But you realize that means that if the scale is right and Outerra supports the ability to collide with models you'll be able to actually WALK inside the buildings? How cool is that? :)

I took a lot of time on Helm's Deep chopping out the ground trying to leave just the structure. Let me know if it's worth it to even worry about that. I figured it'd be a lot easier to "place" into the terrain if there's as little landscape as possible in the actual model.

More coming....

EDIT: Realized models already have their own thread. In the interest of keeping this thread on topic I'll repost my links in the model thread.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 01, 2014, 09:22:14 am
Thanks Morcrist! We can try the Imladris one but it really need a cleft to sit in. At the moment there isn't one on the map. I think I'll use the Lauterbrunnen from a dem and try work that in.

I was trying to source a Minas Tirith myself. This one's not for sale until completed apparently.

http://www.render.ru/forum/images/upload/3272471.jpg (http://www.render.ru/forum/images/upload/3272471.jpg)

You can watch progress here:  http://www.render.ru/forum/viewtopic.php?topic_id=136479&start=3060 (http://www.render.ru/forum/viewtopic.php?topic_id=136479&start=3060)

Time to head over to MineCraft, reg and say hello! :)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 01, 2014, 07:15:44 pm
I DL'ed MT and ran it up on MeshLab. It was a bazillion polys so I tried to cut it down. But I was also doing medem tiles at the same time and my PC got to about 80deg C so I had to cut it off. I think its possible to cut that model down to something which still looks like a good place holder but with 100K or less polys. But its big model to deal with.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 01, 2014, 08:41:40 pm
Do you think it's best doing that in MeshLab or Max?

 I'm probably going to build minas Tirith myself at some point. I can't see anyone volunteering to do it and it's the centre piece of the entire map really. Ennecap was offered $3000 for it and he didn't sell it, so there's no chance I'm paying that. I'd rather do it myself and learn Max in the process. I think I'll be blocking it out to begin with so it'll be possible to model it as a group venture. I have the ref map that he used- spoke to him today- and he gave me some tips. He said that he might struggle to finish his as he's created a problem for himself by not building it modularised. So texturing he said is going to be a big problem. He also said that with a Quadro and 32 GB it grinds to a crawl- He showed me and he was getting about 2 fps as he panned around. In fact I think it'd be better to build something like that in AutoCad or something. I was always going to approach it modularised anyway, down to door frames, windows and such, so that just reinforces my opnion.
 Anyway that's for the future, for now we can trim the model down or use one of the simpler ones knocking about on Google Warehouse, etc. I was considering separating the model into tiers, but it'd still be WAY above the 65K vertices limit or whatever it is for single model in Outerra. I got 2.3 million vertices here- so that's what 700K tris is it? 700K/7 = 100K. Pyton can clarify what that is.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 02, 2014, 05:13:14 am
Yeah, wow. If Outerra only handles 65k vertice models...heh. You know the exporter I found lets you select which portion of the minecraft level you want to export. But still...dang. What's 65k into 2.3 mil? 35? Lol!

You know I'm looking at the exporter and the export options. It's got a checkbox for 'Create a separate object for each chunk'. A chunk in minecraft is a 16x16x256 collection of blocks. My level for Minas Tirith was 512x512 I believe, which comes out to 256 chunks.

I have no idea what the term 'object' means though, in terms of these modeling programs. You should probably take a look at the program (http://jmc2obj.net/). It's pretty nifty.

How can the guy make money off duplicating a copyright protected work anyway? Sounds fishy. You would think the most he could hope to gain from it is notoriety. Which of course could translate for him into the same thing ($).

I really don't like the movie version in any case, preferring something like this (http://corecanvas.s3.amazonaws.com/theonering-0188db0e/gallery/original/tirdawn.jpg), and this (http://tednasmith.mymiddleearth.com/files/2012/07/TN-In_Haste_to_the_White_City.jpg). But of course that's just personal book-centric preference. This (http://tolkiengateway.net/wiki/Minas_Tirith) also contained some interesting info. For example the bottom-most circle-wall was made out of the same black indestructible rock Isengard was made out of. The rest of the walls were white. Nasmith even made a version (http://tednasmith.mymiddleearth.com/files/2012/07/TN-In_Haste_to_the_White_City.jpg) of it with the "correct" walls. Each of the 7 tiers was 100 feet higher than the one below it. It's estimated that the diameter of the bottom-most circle-wall was ~3000 ft.

Anyway, I really want to take a stab one day (in Minecraft) at making it too. Black wall and all. :P

Redrobes, I don't see where you ever responded to my question regarding redistributing the 8 bit greyscale heightmap tiles monks made available to me if anyone ever asks (no one has at this point). Monks said he didn't mind with proper creditation, but that it was a democracy. :)

What say you?



Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: PytonPago on February 02, 2014, 07:41:08 am
I think the "object" refers to "meshes". Actually, the 64k vertices limit is for individual meshes - of witch a model can have even thousands , so you just have to split the model into multiple, 64k max, meshes, then OT importer can handle it.

As for MT - thanks for interesting reading and images. Whyte a near half kilometer, it should be considered a skyscraper ... (our country highest (AND 2 OR 3-TH LARGEST IN INNER CAPACITY) castle is Orava http://static3.depositphotos.com/1007429/239/i/950/depositphotos_2394354-Orava-castle-Slovakia.jpg (http://static3.depositphotos.com/1007429/239/i/950/depositphotos_2394354-Orava-castle-Slovakia.jpg) ) and it has just 112 meters above the river (starting from 20 maybe). Hard to grasp the size and internal structure ...

There is also an interesting Hungarian site white some city schematics here : http://olvasas.opkm.hu/portal/felso_menusor/konyv_es_neveles/a_szimmetria_szerepe_a_tolkieni_utopiaban (http://olvasas.opkm.hu/portal/felso_menusor/konyv_es_neveles/a_szimmetria_szerepe_a_tolkieni_utopiaban)
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 02, 2014, 10:58:58 am
 I think he's primarly doing it for his portfolio. He was bemoaning the Ukrainian economy- a lot of political unrest on those parts apparently. He's doing freelance architectural visualisations but want to get into compositing. He wants to upsticks to Canada. But yes, that's the point, it's very obviously based on the movies. You can't make money from this stuff, whether it's based on the movies or not...It'd be awesome if he offered us a 1/4 res model or something. I'd even offer him $ for that.

 I mailed the guy who has the model for the original Minas Tirith project on CGTalk. He's selling a render of it on TurboSquid for $20 and there's a comment form the artist saying "I couldn't upload the model as it's 180 mb" or something. That sounds to me like a back door, telling folks, I have the model, contact me and offer me $.  ;D  I could be wrong, but it seems a bit pointless to be selling a single render (not even multiple views of the model) on there....

 Nah, in the end, the only way is to do it yourself, although I can understand you wanting to use our terrain Morcrist- just like we want to use someone else's Minas Tirith heh. Anyway, the terrain's far from perfect so you can go on to still do a lot of alterations and tweaks to it. I'd prefer it everyone got together and made one map, one Minas Tirith- all to shockingly good levels of expertise. That's the ideal, but the world ain't like that. And I guess a large focus of talent and interest like that would more likely invoke a C&D anyway. I also like plurality as well, I don't like it when a group of people start to think that their's is the only way to do something. LoTR is a book, a masterpeice, but it's not brain surgery- what I mean is, have a perspective, it's not going to save you life on the operating table. So have fun with it. The films are the films and the books are the books. They're different. I love the films. I love the books more. But bear in mind Tolkien couldn't do everything to the standards/knowledge he wanted to.
 I think the MineCraft community is pretty cool for their openess to share their models. I guess that there's an element of well they're not full CG quality models, so selling them is not really an obvious option. Not that I'm criticising the modellers on there- I've seen loads of freaking amazing models, not least the M-E stuff.
 I've always liked Naismith. I think out of all of the artists I've seen he has this kind of old story book feel to his work. CG is not even on the horizon. I think stylistically he's my favourite Tolkien artist.
 Yes, Fonstad had the city walls at about 945m, the films at 1207m. I think Nasimith's is at the Fonstad end, probably wider. Fonstad used the tower of Ecthelion's width as compared to the Tolkien sketch I think. She imagined that, knowing the height of the tower, a width of anything greater than 150 feet would make the spire less a "spike" as described, but more squat. Extrapolating from that, she got the overall city dimensions. It's open to question especially since she's going off a really rough sketch, but it's a sound process.
 At 1000 feet high that's some impressive bit of masonry :-D that pic Pyton recalls a castle I went to recently in Yorkshire called Skipton castle. It's a very similar outlook, although it's not quite as high as that- maybe 150 ft. But the views from the castle rooms were amazing. Imagine what a 1000 feet loks like??  :o
 I like the prow, the way it goes up at the end, more for artistic, naturalistic reason. It's more romantic. I like the prow in the films as well for practical reasons- you would really want to be able to look out over the lands from a good vantage. I think the reasoning would be, if they're capable of mining and building that city, then they'd be capable of levelling off that prow at the end for better views.//ah I see it's a man made structure so that's irrelevant.
 Yes the black wall needs to be in there obviously!

We'll either have to cut the model up into pieces or use a simpler one off Google Warehouse or something. I'd rather try a combination of simplify and cutting it up. I'll try a mesh decimation later in Max, see if I can get anything decent out of it.
 I think distributing the terrain is fine- I think make it clear on the link page that an exchange of models for terrain is the idea here, or that we're looking for models. Maybe some folks will come over an offer something. ;)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 02, 2014, 04:49:59 pm
Exported another Minas Tirith with the "create object for each chunk" selected. Let me know if it does anything different.

Here. (http://morcrist.com/files/minecraft/MinasTirithChunked.zip)
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 02, 2014, 08:19:00 pm
Hey Pyton, first map I've ever seen of Gondolin...!

Great stuff! There's more vertices but it's a bigger model. It chunks them all up just nice. Max runs out of memory if I try to open them all up as separate chunks. I can open them up as a single mesh, and in two halves as separate chunks.

 That links looks good btw. With me not having MineCraft I need a way to convert the Minecraft format. I'm guessing that does the trick. I managed to download a couple of models without registering. Gonna try regging again shortly.

Models we need most:

Barad Dur
The Shire...something, anything...
Bree would be nice.
As would Weathertop. A single tower will be good enough for that.
Rivendell is a tricky one.
Argonath. I've seen models on Google Warehouse I'm sure.
Isengard. That's a popular one. Preferably one with the Ring, not just the tower.


We'er just finishing off the texture for the beta 1 biome run at the moment. Should have a new map release for Outerra soon!

monks

Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 02, 2014, 09:18:49 pm
Lol! So much for keeping this thread on-topic.

Isengard (http://morcrist.com/files/minecraft/Isengard.zip).

I chopped out most of the terrain. The piece missing from the ring is where a mountain spur jutted inside it. The inside of the ring also represents an earlier period from the books. Much cooler period, but I can find one without vegetation if you'd like. Probably need to chop it out anyway since Outerra has awesome vegetation. It was just the one I had so I'll get another soon.

EDIT: Yeah sorry. I'm an idiot. I'm cutting out the vegetation now.

When can someone get one of these into Outerra so we can see if they're navigable? As in, walk-around-in-able?

Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 02, 2014, 10:05:01 pm
Outerra has a 64K poly limit so either it needs remeshing to that or it needs cutting into chunks. I had a look at Minas Tirith and it was cool but big. I tried to make it smaller but I had to bail on it. I will try again tho. I have never yet imported a 3D model into outerra so I need to learn this too. I think maybe you need the full demo but I am barely able to run the demo on my machine. In fact I have not yet ran MeDem with these new biomes..... sad huh !!!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 02, 2014, 10:20:51 pm
Time for an upgrade!

Importing models is not that difficult but I only ever tried with single meshes. I think Pyton might be the man for getting multiple chunks in there.
 Going to have to learn though. I've got ideas about using City Generator. It's the only fecking way we're going to fill this landscape LOL

We need something for the Shire really if were starting the vid from there. I found this- looks interesting: http://www.mcmiddleearth.com/media/michel-delving.7/ (http://www.mcmiddleearth.com/media/michel-delving.7/)

Once we get this terrain settled in next run, and get this biomes map spot on, I'll be getting down and dirty with Max. Minas Tirith first up I think. Gonna have to move a crate load of crap off C drive, so probably move Make over onto 1 TB E drive or something. Plans, plans...
Can't wait or Occulus Rift btw- those mountains are going to look massive in that...that's a really popular vid for Outerra- I added the head up about OT support in our YT channel today- I think people as pretty stoked about that.

 GTS code for generating glaciers and lava flows would be cool to try out as well!

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 04, 2014, 05:01:21 am
Is putting a model into Outerra something a layman could do (i.e., myself)?

If so could you point me to a resource describing the process? I'd really appreciate it.

Nevermind, found it here (http://forum.outerra.com/index.php?topic=1778.0)!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 04, 2014, 04:03:31 pm
Go go go!!

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 06, 2014, 03:24:51 am
Hey monks and/or Redrobes, I was wondering if you could help me out with something. I downloaded and installed GIMP in the hopes that it could let me work with larger image sizes, and it did.

So I made a new 20,000 x 20,000 and started to import the tiles you sent me starting with x=0, y=0 in the top left corner, but I started noticing some odd things. Pics are worth a thousand words, or so the saying goes so here:

(http://morcrist.com/images/minecraft/topleft_thumb.jpg) (http://morcrist.com/images/minecraft/topleft.jpg)

Problem spot #1: the black/eroded border along the bottom of y=1. Why is that there?

Problem spot(s) #2: y=2 and y=3 tiles don't match up. I circled some of the more glaring examples.

EVERYTHING matches up perfectly left to right, on the x axis.

Anyway, hope you can shed some light on what's going on. I'll try and finish mosaic'ing the tiles so I can get a look at the whole thing.

Thanks in advance!

EDIT: Here's a jpg of the entire thing without green/red lines so you can more easily see the non-matching parts: http://morcrist.com/images/minecraft/topleft_big.jpg (http://morcrist.com/images/minecraft/topleft_big.jpg). You may want to right-click and save-as that sucker. My Firefox just crashed when I tried to load it in the browser. :O
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 06, 2014, 08:56:40 am
Have you opened 3,2 and checked it prior to Gimp?

 It could be the tile export from World Machine was corrupted. Very occasionally it will do something like that. Looks like tile blending.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 06, 2014, 10:51:07 am
Monks, where are these tiles from ? Are they pre-blended set or something. It looks like some tiles from WM but it doesn't appear to be anything from the make build since GTS doesnt do that kind of erosion. We will have to sort out some extra export stages for people and run a texture pass which renders out the height as a linear ramp to PNGs. Thats about three of these exports we need now.

Anyways, the answer from me is I don't know whats going on there !
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 06, 2014, 10:56:20 am
I'd just got out of bed when I looked at these...I can see it's either Gimp loading them incorrectly or WM export. Have you tried opening say tile 3,2 in Photoshop? It could be WM or  Gimp. I might still have the tmd that I outputted the tiles with..I'll have a look.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 06, 2014, 12:49:27 pm
Have you opened 3,2 and checked it prior to Gimp?

It could be the tile export from World Machine was corrupted. Very occasionally it will do something like that. Looks like tile blending.
Oh, yes. A week ago when I first got them I opened them up in Paintshop Pro and absentmindedly noticed some of these "artifacts" but didn't pay them much mind not having much experience with this kind of thing. Plus since the tiles were so big I was only opening up one at a time (i.e., the Lonely Mountain tile, the x=3 y=4 tile for Moria/Isengard, etc.)

It was only in the last few days when I started trying to mosaic a couple of them together that it became readily apparent something was going on. I think you hit the nail on the head when you said, "Looks like tile blending." If you look at the bottom right corner of the map you can see where it looks exactly like that. There's a thick river that looks blended into the mountains, not eroded out of it. And now that I look closer at the y=2 tiles I can see some more evidence of what looks like more than one thing getting blended together.

And for the record, I am loading these from the original BMPs I downloaded from google. I stuck them all in a directory and copied them out to another directory I was doing things to so these are the originals. Google zipped them up before I could download them, but I don't see how that could have such drastic effect on them?

I really appreciate you taking the time to check this out!

EDIT: Kind've changing the subject...I don't know if you guys are familiar with Ventrilo (http://ventrilo.com), but it's a simple voice chat program I use with some friends/family. It also has text chat which we actually use more than the voice part. Anyway, I've got a voice chat server I pay for that I'm pretty much on the entire time I'm at my computer. I started using it back when I played WoW, plus it's pretty much the only way I keep in touch with one of my younger brothers (Nazghash).

I'd love for you guys to get on sometime for a chat! You'd have to download and install the client, but it's a very simple/unobtrusive program. I'm GMT -6. If you guys are in England (like I suspect) timing the chat may be our biggest difficulty. :P

If you do decide to get on, my server info is: morcrist.typefrag.com, and the port is 49450. There is no password. Get on and hit the Chat button on the right to get in the text chat part. One of its main attractions is the ability to paste links when you're talking.

I leave my computer on a lot, so if I'm on but unresponsive remember that and the fact that I'm GMT -6.

Anyway,
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 06, 2014, 01:16:15 pm
Problem with WM bmp tile output confirmed.
 It did the same thing again with another output. It could be any of the following:
WM version. I can try another.
flipper device which flips the tile vertical axis. Necessary because of the import tile set. There does seem to be an issue within WM with tile coherency after the flipper in the network. Again, this bug could be version dependent.
bmp format. try another.

The worst case scenario I think is it's the flipper because that will create work...I could run the perl flip script I've got here actually which would do it in like a milisecond.

i'll take a look at it later on.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 06, 2014, 04:48:33 pm
I think there are 2 separate problems here making things a bit confusing.
The tiles exports are broken. Both in bmp and tga. Strange that. That suggests it's not the tiled export causing the problem. They can't both be broken- very unlikely.
The flipper device creates incoherence in the viewports but I'm not sure if it's responsible for the tiles being broken, but I suspect it is. So I'm going to have to export a tiles set without the flip on it and check them.

//the flip is creating the errors. I just exported an unflipped set as bmp and reimported. I have no problems with the tile coherance in the vewiport.
 I don't think I can run the script on them because the flip here is not the same thing. This is individual tiles being flipped, not a tile set position.
 I didn't export them with the flip and to be honest I get kind of lost with what workflow needs what flip. So I think this set will be ok.
 I'll upload a new set later.

 Format must be 32 bit RGB or something...I have no control over that.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 07, 2014, 03:03:38 am
Thank you so much for looking into this for me. If I have to manually flip each tile, no problem.

Eagerly awaiting the new set, at your convenience. No rush!

Take care.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 07, 2014, 10:38:51 am
Let me know if you have them downloaded so I can delete them. My GD is maxed out :)

 Next set...soon as we can. It's already closer to where it needs to be.......

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 07, 2014, 12:52:50 pm
I'm sorry, but now I'm confused. You had said, "So I think this set will be ok. I'll upload a new set later." I then replied that I was eagerly awaiting this new set and then you said, "Let me know if you have them downloaded...." And finally you said, "Next set...soon as we can."

Lol? I checked the GD and it appears to be the same, old tiles? Last modified 1/26.

If I'm just an idiot, then yeah. I apologize! :P I downloaded those the first day you had them up and you can go ahead and wipe them.

If I'm missing something, please let me know.

EDIT: Now I see that I am in fact an idiot. I missed the new email. Derp.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 07, 2014, 01:02:16 pm
 ;D ...Minecraft set is up on GD...ME-DEM release terrain in the works.
deleting...deleting...now!

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 07, 2014, 01:29:10 pm
Gotta' run to work. Quickly looking at a couple of the ones that weren't working LAST set...looks good. I'll do some mosaic'ing when I get home, Lord willing.

Thanks again so much guys!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 08, 2014, 06:16:54 am
Got half the map mosaicked and it looks awesome. Definitely fixes all the problems, monks. Thank you SO much.

I made two 20k x 20k maps so far out of (x=0-3; y=0-3) and (x=0-3; y=4-7). Got to get to bed for now.

Thanks again!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 08, 2014, 10:15:16 am
 ;D Awesome...Middle Earth comes to life!

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Fornad95 on February 10, 2014, 03:51:06 pm
Hey - I've got a similar project going over at www.reddit.com/r/worldpaintingarda (http://www.reddit.com/r/worldpaintingarda). Would it be possible to get a hold of either the GIS files or a 10k heightmap?

I'm going for a more 'hand-painted' method, but some of the formations would definitely be useful, as well as seeing how you guys handled some of the areas that aren't that detailed in the maps.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 11, 2014, 04:42:10 am
I've uploaded the tiles monks provided to my website, here (http://morcrist.com/files/minecraft/ME-DEM.zip). They're greyscale BMPs, but they're still 24 bit at present due to the way his World Machine exported them or something. First thing I do as I open each 5000x5000 bmp is turn it into a true 8 bit greyscale. When I've gotten them all converted I'll re-upload the much smaller files. Right now, zipped they're ~170 MB.

Just remember that if you use them, you need to give credit to the ME-DEV ME-DEM project and/or monks/Redrobes. Their "official" website is here (http://me-dem.me.uk/). Correct me if I'm wrong, monks!

EDIT: ME-DEM project, sorry. At least I got the link right. Heh.

Anyway, nice screenshots man. Can you tell me what shader you're using? I've got OptiFine installed and SEUS's shaders. I'm using an admittedly older GeForce 9800 GTX+, but still...I can't run anything more than SEUS Lite. It sucks! Any advice would be GREATLY appreciated.

Take care.

EDIT: What kind of rig are you using? I've only got an 8 GB Dual Core Pentium. Takes a long time to work with one tile. Heh. Are you able to work on the entire 50,000x45,000 map in World Painter? Or do you have to work on chunks of it?
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 11, 2014, 10:37:46 am
Please do not make available our data at full resolution. Our policy is not to give it out- It represents blood, sweat and tears of years work. We are fine with helping people and giving out lower resolution data. For eg, 10K @ 16 bit grey, or 40K @ 8bit. We are fine with you making it available at those resolutions. Hence we don't give out GIS data either.

 Nice project Fornad. Some of the shots are pretty sweet. Our terrain is not canonical. It's not based on the Tolkien map. It's based on the ICE role playing map which has some innacuries. So any GIS data would not be useful to you anyway.

 It's possible that I'll create another map. With the new workflows I've recently developed, the software advancements over the years, and the raw materials we now have. I could do it an order of magnitude quicker. Probably less than a year. Using this workflow it's now possible to create correct hydrology at every point on the map, not just on major rivers, and for the terrain to be highly scalable to higher resolutions, still preserving equal detail, independent of satellite dem resolution.
 
monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 11, 2014, 12:27:32 pm
Please do not make available our data at full resolution. Our policy is not to give it out- It represents blood, sweat and tears of years work. We are fine with helping people and giving out lower resolution data. For eg, 10K @ 16 bit grey, or 40K @ 8bit. We are fine with you making it available at those resolutions.
I panicked when I saw you wrote this because I don't want to do anything to offend, but my understanding is I'm not doing anything you said I couldn't do.

I don't even have the data at full resolution? What you gave me was 40,000x40,000 at greyscale, so 8 bit. The actual BMPs you gave me may have been 24 bit BMPs, but they only contained 8 bits of data. In other word they were 24 bit image files with only a 256 color palette.

Are we good or do I need to take down my links?
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 11, 2014, 12:28:42 pm
Credit is to ME-DEM (not DEV) project. Thats the Middle Earth Digital Elevation Map project.

I think the data you have is 8 bit greyscale even tho the images were 24 bit. I think the RGB were all the same. I think at some point we will export a lower res 32bit DEM and a 10K greyscale PNG as standard but the base DEM is changing a lot at the mo and its not settled so its not great having various versions and copies of something like that floating around.

Just double check with Monks tho. I write the scripts and tools but I always think of the main map as his work.

EDIT - oh yeah he said 40K @ 8bit is ok in earlier post.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 11, 2014, 12:49:58 pm
Nah, it's ok Morcrist I just thought that maybe I'd given out the full res data by accident. I don't mean to be mean spirited mate, but over the years we've received very little help whatsoever from anyone, and had some rebuffs too along the way. What's mine is mine, etc. As I said the GIS data wouldn't be useful anyway, but Fornad can have the 16 bit grey @10K no problem. In fact I could separate out the mountains for him if that would be more useful. @Fornad, you could shape the mountains to your map by using the warp tool (or better still puppet warp tool in Photoshop). I can also give you the topo map we used, and you could try interpolate the extra information into your map.
 Our map is not canonical as I said, but it's really been a fascinating journey and the stuff I've learned and people I've met, has been fantastic. All power to your elbows- I just wish you could come over here and team up, but hey that's no problem either! I think both of you will make inspiring worlds- Fornad is already well on his way.

 Robes always says that, but we're a team! I could never do the things he can do with code! :)

 This new workflow is potentially a vast improvement on what we could do (IF we can pull it off), but it's not as elegant as David Genevaux's (SIGGRAPH 2013 - Terrain Generation using Procedural Models based on Hydrology (http://www.youtube.com/watch?v=JCsj0v-wmIM#ws)) I'm talking to this guy and believe me, we are in for some VERY interesting times ahead.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 11, 2014, 01:17:31 pm
Ok, whew. Good to know. And you don't come across mean spirited at all. I completely understand this project is like your baby and I feel absolutely honored that you've given me anything at all!

Until Mojang updates Minecraft to support more than 256 height, I don't think anything greater than 8 bit resolution would be helpful.

Thanks again guys.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 13, 2014, 07:14:16 am
Just an update.

10,000 x 10,000 Greyscale (http://morcrist.com/images/minecraft/10K.jpg) of the entire 40k.

With my outdated software I can only make up to 32,768 x 32,768. And even then I don't have enough memory for it. With GIMP 2.8 I was only able to create two 40,000 x 20,000 images. When I tried to stick them together to make a 40,000 x 40,000 I kept getting crashes.

World Painter Image (http://morcrist.com/images/minecraft/WP10K.png) of the same greyscale exported as an image. I think I set the dirt level way too high, but it's only at 40 so *shrug*.

Meanwhile I talked with bearman from the Massive Middle-Earth Map project (the one who inspired me to start all of this). We got on Skype and he actually showed me his desktop as he played with the 40k data in World Machine. Where of course it looked sweet. :) While talking about exporting from World Painter he noted that there're two Offset fields you can use which ended up making it painless to break the exports up into smaller (10,000 x 10,000 for example) chunks. Now I just need to start importing/exporting in WP and get this puppy generated in Minecraft and uploaded to my server.

Take care.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 13, 2014, 11:13:03 pm
It looks cool. It's interestin to see and alternative view of it in another app. I can see so many problems with it though- the terrain modelling..hehe...but that's just me. People in the world will not notice such things! :)

 Update with the biomes: http://www.me-dem.me.uk/galleries/Outerra2/gallery.htm (http://www.me-dem.me.uk/galleries/Outerra2/gallery.htm)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 19, 2014, 05:54:14 am
I've finally managed to get some work done. I finished converting the original 40K 8 bit data you gave me from 24 bit greyscale images to 8 bit. They take 3x less space on disk, ~50 MB less zipped.

The link (http://morcrist.com/files/minecraft/ME-DEM.zip) is the same.

I also finished running my Highlight/Midtone/Shadow color filter on all of the tiles AND making an 80K version of them. I simply converted each 5K tile to 10,240 x 10,240 - the extra 240 is so it fits into World Painter perfectly, which uses 128x128 tiles. The color filter, again, was to stretch the water/plains down while keeping the mountains high. My water level in Minecraft is 17 now, plains are down to about 30. It looks nice, image wise and in-game.

My new cpu finally arrived. My gpu should be here tomorrow. I look forward to increased rendering/import/exporting capability! Heh.

And speaking of which, yeah. That's what's next. Importing and exporting all the tiles to Minecraft.

Take care!

EDIT: Now running with my new cpu and gpu. Exports from World Painter seem to take half as long now, down to 20-25 minutes from 45-50. And I can run SEUS shaders in Minecraft!!!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 20, 2014, 06:50:52 am
Monks, Redrobes, what did you guys use for overlays? I'm wondering if it's just me, or if it's the map I'm trying to use. I can't for the life of me get my overlay to line up with the terrain.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 20, 2014, 08:04:22 am
I did tell you that the map was not canonical..:) I'll upload the map we used later for you.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 20, 2014, 04:19:01 pm
If it isn't too much trouble, you once mentioned river/forest overlays? If you had those on a per-tile basis (the 5K tiles you sent me) that would be stellar. That's what I'm trying to use the overlay for in any case, painting the rivers and forests.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 20, 2014, 05:40:20 pm
yeh, no probs...topo going up...you'll have to cut that up though. It's 20K- and probably non square- something like 1 pixel out- 20,008 x 20,007. I can export tiles from Global Mapper...I tell you what I'll do...I'll export the files again from Global Mapper and then tile them for you in World Machine in a new project ;)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 22, 2014, 02:46:24 pm
yeh, no probs...topo going up...you'll have to cut that up though. It's 20K- and probably non square- something like 1 pixel out- 20,008 x 20,007. I can export tiles from Global Mapper...I tell you what I'll do...I'll export the files again from Global Mapper and then tile them for you in World Machine in a new project ;)
Why would you export the files again?

The topo should be fine. I'll resize it to 20k even, cut it into 2.5k x 2.5k pieces then resize each to 5k to match the 5k height tiles you gave me. That should work out fine, right? I mean, assuming it's the same topo you used to guide the terrain creation. :)

Take care.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 22, 2014, 08:26:41 pm
I've got the following at 40K (5 x 8000) which will be going up over the next 2 weeks...it's a bit of a job, so I have to run the build and then the upload overnight- so two nights per map.
 I can't really build the WM project, what's more the maps in there are not all at 40K anyway- we use a make system so some things have to be resampled. I've also spotted some important missing rivers in our river map- which will be corrected next time.
 Topo
 Rivers
 Primary Roads
 Secondary Roads
 Tertiary roads
 Forests
 Wetlands

 I'll upload Rivers overnight.

 If you want the maps at 20K res instead, let me know. Rivers really have to be 40K though fore us- but if you're only using them for a visual guide you can use 20K ones..

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 22, 2014, 09:36:27 pm
Whoa! You are too good to me, sir. I thought you might have meant something like this, but I didn't want to get my hopes up!

As far as what res, as always whatever is easiest for you. I'll upscale/downscale it to fit my needs and, since this is Minecraft we're talking about I don't think it's going to matter at those resolutions. :P

Thanks so much, monks.

Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 23, 2014, 12:00:15 pm
I hope this is ok. It's 8 bit grey bmp. I've still got the RGB files just in case these are not good. I've had to use Photoshop automation, which is new to me so I'm learning something in the process- should've been using this years ago!

http://www.skindustry.net/medem/files/Temp/Morcrist/Rivers.7z (http://www.skindustry.net/medem/files/Temp/Morcrist/Rivers.7z)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 23, 2014, 01:49:02 pm
Hey monks. Now I know what you've been going through...I ran out of C: drive space unzipping these! Hah! All these damned image/map files! Heh heh.

Anyway, I opened up x2y3 (the Shire area) to check out what it looked like overlayed on the 40k 8 bit heightmap at that location...and it didn't line up.

I don't have time right now to stitch together the rest of the bmp's to see what's going on but just thought I'd let you know before you do anything else.

When I get home I'll dig into and see if I can tell what's wrong.

Base Tile @ x2 y3 (http://morcrist.com/images/minecraft/x2y3.jpg).
Rivers @ x2 y3 (http://morcrist.com/images/minecraft/x2y3rivers.jpg).
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 23, 2014, 03:17:40 pm
Your heightmap is 5120 x 5120. I'm guessing you made that change your end. The two tiles don't correspond. Is it because each one of your tiles has the extra pixels and is throwing the alignment out? I gave you the original heightmap at 5000- just checked the source tiles.
I wasn't aware you'd changed it.
 If you've still got 8x8 tiles, then the tile should fit if scaled up to 5120. How many tiles do you have?

 I have about 5 TB hard drives full of this stuff...I've got duplicates though, but there's vast amounts of it..:p

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 23, 2014, 08:26:40 pm
Oh yeah, first thing I did was resize it. Well that's not true. First thing I did was copy it to the heightmap and notice it had a 60 pixel border all the way around lol. THEN I resized it! :P And I resized then 5000 x 5000's on my end because World Painter works internally with 128x128 pixel tiles. After importing I'd end up with a water border 8 pixels wide on the right and bottom of the maps that need to be removed before exporting. Instead, I just upsize each 5k tile to 5120 x 5120 before importing and it comes out perfectly to 40x40 World Painter tiles, no border to get rid of.

Anyway, check this out. I turned the rivers blue (to aid visibility) and laid them on top of the heightmap, same tile. x=2 y=3.

(http://morcrist.com/images/minecraft/x2y3combinedthumb.jpg) (http://morcrist.com/images/minecraft/x2y3combined.jpg)

You can see from the terrain that the rivers don't have any correlation. I tried flipping and mirroring the rivers but no luck matching them  up.

I just got home so I'm going to check some of the other images out.

EDIT: Ok, I got lucky. Found the problem on the next image. It's offset by one in the y direction. I'll upload to show you....

Base x=2 y=3 tile (http://morcrist.com/images/minecraft/x2y3.jpg).
River x=2 y=4 tile (http://morcrist.com/images/minecraft/x2y4rivers.jpg).
Combined (http://morcrist.com/images/minecraft/x2y3x2y4combined.jpg).

So the rivers for x2y4 match up with the heightmap for x2y3!

Any ideas? I'll verify a few more and let you know.

EDIT: Ok, bleh. This is just weird. So far RIVER x0y3 fits with HEIGHTMAP x0y4, and RIVER x0y4 goes with HEIGHTMAP x0y3!

Got it. The y values are backwards. So while the heightmap tiles go from x0y0 to x0y7 and then x1y0 to x1y7 etc., the river maps go from x0y7 to x0y0 and then x1y7 to x1y0 etc.

So no problem. I can just rename the files and I'm good to go! Should work out well. I don't know what this means for your exporting workflow though. Heh heh?
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 23, 2014, 09:19:38 pm
Oh balls...you should have mentioned the 60 px border.........I'll have to track down the pre-make terrain tiles. I'm not sure if I still have those...It should be a 120 border actually, since make frames the map with a 120 px border. @5120- yes it is.

 So the tile set should cover 120 + 5000x8 + 120 = 40,240 px. Can you verify that?

 Make always puts a border around it. It's a bit of a pain, and we're moving away from it- if we've not already done so for the next release.

 If I don't have them, one option is to build the World Machine terrain again. That ties my computer up for about 14 hours, and that's IF I'm lucky it builds first time. It crashes sometimes. I did have the project on my SSD and it proved to be a bit unstable- don't ask me why. It's now on C drive so it should be more stable for builds. But I think lack of hard disk space also makes it more unstable.

 There is another option. Merge the tiles in Gimp or PS, cut the border off and re-tile. The better option would be to re-tile them in World Machine because PS doesn't have tiling features, but editing them down into a workable input is going to be a royal pain...... Does Gimp have tiled output?...I think it has right? If you merge the tiles into say 4 x 20120 pieces. Edit them down individually to 20K. Save them out. Open them up as a single doc, then tile them out. That should solve it. That method might be easier on your system anyway if you can't do it all in one doc. This way you don't lose the work you've done so far.

 I'll let you know later if I still have them...WM building Helms Deep right now..

 monks



 
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 23, 2014, 09:33:22 pm
We have a lot of issues with this kind of thing. WM exports with Y going up and we want Y ascending going down. So we have to flip the order sometimes. Usually the tiles themselves have the correct pixel direction but the naming is opposite the pixel direction. In our system we have top left tile as 0,0 and top left pixel as 0,0 within each tile. If you have a 2x2 or a 5x5 set then you can manually do it but if you have more then its best to have a script to rename them.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 23, 2014, 09:46:18 pm
Yes, WM naming is quite annoying, and no other software I use uses that convention. It's possible that the tiles were flipped but I seem to remember you said it took you a short while to work out which way they were oriented, so I presumed you'd solved that. You'd know for sure if you hadn't because you can see them all in Gimp! We use a perl script here to flip the tiles from WM going into make, and I could always mod it to operate on bmp format. IF I find the tiles.
 I think the best bet is to edit them yourself, now you've got them oriented ok, and done the post processing you did on them.

hmmmm...don't think this will work either since Make has resized the tile set after adding the border?..is that right Robes?

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 23, 2014, 10:06:29 pm
Oh wow. I do apologize. It seems I've completely given the wrong interpretation of the problem. My mistake!

Let me clarify. The river tiles were fine image wise. They were all 5000 x 5000, the same as the heightmap tiles.

The only problem was what Robes mentioned, the y order being backwards. I will rename them. It won't take long.

Again, I apologize! I remember mentioning in one of my previous posts that I'm an idiot, so it's not like you haven't been warned.

:P

And if you care, what I MEANT was...since the heightmaps you gave me were only 5000 x 5000 and World Painter (Minecraft world manipulation program) expects things to be imported in nice 128x128 pixel chunks, I resized each 5000 x 5000 heightmap tile to be in multiples of 128, or 5120 x 5120. Nothing more than that really.

So yeah, river pixels are fine just the naming was backwards! But again I'll just manually rename them and I'm good to go.

No need to tie up your machine doing anything to "fix" this. Problem solved!

Sorry for the confusion!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 23, 2014, 10:12:19 pm
Heh heh - So you know that 120 pixel border.... well we have tiles of 40960 with a 40000 image in them so we have 280 pixels on every side spare. Thats why we have a border. So we have 40x40 x 1024x1024 tiles not 40x40 x 1000x1000 tiles. Its a lot easier working in 2^n values so our dataset has to fit into the middle of it. Its not "Make" that needs the border but the apps that need 2^n sized tiles to work with. So your tiles are 4096+1024 sized or 5120 which is why you have the spare 120 pixels to manage ! Made be chuckle a bit there you adding it back on ;)
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 23, 2014, 10:19:12 pm
 ;D ...ahh. So I did have those tiles then. I was pretty sure I have both the input and output to make, so I would have given you the input (which has just come from WM). I'll flip the tiles I give you in future.
 If you haven't already, I'd recommend getting into the idea of using scripts. You might already be au fait with them, but it can save oodles of time. If you need any advice in that regard Robes is your man. I've got scripts here. I do have another option here now of using CS6 automation which can record operations. Will be useful for tile operations. i'm using batching to resize and rename images for the website. In fact, I really must try track down a Photoshop tiling script, or something. Surely there must be something out there.

//thanks to Robes for clearing that up- yeh, of course, it's the target apps ;)

 
monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 23, 2014, 10:22:47 pm
Scripts - yeah good point dont I have that exact script around here....

Ahh yes - lets see how this forum handles zips...

EDIT: You can use a text editor find and replace on the name or the directory to make one that is suitable for you.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 23, 2014, 10:31:48 pm
I'll flip the tiles I give you in future.
If by flip you mean rename the y axis, sure. If by flip you mean pixels, please no! They were perfect as-is.

And Robes, yeah. Heh. That is kind've funny if I'm understanding you right. Although the heightmaps monks gave me were 8 x 8 tiles of 5000 x 5000 pixels each so I don't know about any border...? If you meant that he gave me the un-bordered tiles and then I had to add the border back on, then yeah that's a lol for sure. But sure enough, the app (WP in my case) apparently wants 128x128 multiples and it was easier just to resize the dang 5k tiles to 5120 than to cut out the extra bits that WP generated on import.

Checking out Robes' zip now....

EDIT: lol yeah, that's exactly what I was going to need to do by hand and even addresses the "how do I rename y0 to y7 when there's already a y7!" problem I knew I'd run into. Heh, thanks!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Redrobes on February 23, 2014, 10:36:42 pm
That script is right for you (prob need a name change on the tiles tho). And yeah my GTS needs something like 64 pixel modulo because it works on sub tiles within one tile you load in. So I think WM probably has a 40K set and Monks exports for 5K exact but we add the same 120 pixels on as an offset to them all so that we have 8x8 x 5120x5120 or similar in the start of the make system. I developed a tile cutter which makes tiles out of other sets of tiles so we actually cut 1024 tiles out of the 5K set I think. But its the same principle - instead of a resize we keep pixel perfect but add the same width in border pixels.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 24, 2014, 01:17:07 am
Got NW corner of the map exported (16 5k tiles). This is just my first pass. Nothing except heightmap/ocean.

(http://morcrist.com/images/minecraft/mapNW5k-thumb.jpg) (http://morcrist.com/images/minecraft/mapNW5k.jpg)

48 more to go!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on February 26, 2014, 08:56:33 pm
Monks the river tiles work wonderfully! Thank you so much.

Any luck on the forest tiles?
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 27, 2014, 11:52:35 am
I'll try get them out over night mate. Had a busy few days. ;)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on February 28, 2014, 12:54:57 pm
I did run it overnight but there are some strange artifacts in the conversion. So I'll have to convert them before I tile them. I'll run them again tonight. ;)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on March 01, 2014, 11:10:16 am
Forests are up, you should get a heads up from Google. The tiles are not flipped since Robes gave you the script. Let me know if you have any problems.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 01, 2014, 12:57:08 pm
Looks good. Thanks monks! This will sure make painting in the forests much easier.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on March 01, 2014, 02:37:52 pm
What I'd do Morcrist is multiply the masks with the local terrain steepness...that way you'll get more realistic coverage on the ground- it'll make things look more natural.
 I'll be able to get out the wetlands, roads, etc. For some reason the tmd was taking an aeon to build, when really it should be like 15 minutes. I've got an automation droplet now in Photoshop that converts them all to greyscale. :)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 02, 2014, 01:59:42 am
What I'd do Morcrist is multiply the masks with the local terrain steepness...that way you'll get more realistic coverage on the ground- it'll make things look more natural.
What do you mean? The river and forest masks? World Painter is limited in what it can use when importing, and/or what you can apply to it while it's importing.

I've got a lot of control over my brush though, like when I'm painting forests. I can tell it to only apply trees when the angle is less than XXX, and when the height is less than YYY. This will simulate the fact that trees don't grow on steep slopes and they don't grow above the treeline.

I've also worked out a nice way, I think, to do the rivers. I've started doing it, it's labor intensive, but it looks ok in game. I'll get some shots up of some rivers/streams I've done eventually.

Thanks again, guys!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on March 02, 2014, 09:50:24 am
Yes, that's the same thing ;) Rivers is always time consuming......... :P

 Ok wetlands are up. Next on the list are lakes and roads.

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on March 03, 2014, 12:56:42 pm
Lakes are up mate!

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 04, 2014, 01:08:13 am
Got 'em! Thanks!

What's that leave? Roads?

In a previous post you mentioned a topo layer? What's that?
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 06, 2014, 06:57:44 am
Been working on workflow lately. Got some stuff ironed out.

Got the left half of the world mapped in dynmap, then I stitched them together to make this:

(http://morcrist.com/images/minecraft/map-thumb.jpg) (http://morcrist.com/images/minecraft/map.jpg)

Which is still just basically raw terrain. No rivers, no forests, etc. I started experimenting with the Gwathlo and I like the results so I'll do the rest of the rivers the same way and see what happens.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: monks on March 06, 2014, 09:46:28 pm
Looking ace mate!

The topo is a big download:
http://skindustry.net/medem/files/Temp/Morcrist/Tiles.7z (http://skindustry.net/medem/files/Temp/Morcrist/Tiles.7z)

monks
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 06, 2014, 10:07:31 pm
Wow that is a big download. Grabbing it now.

Man, file naming conventions has been a huge PITA for me doing all this image manipulation. The way an image program will make a series of files named tile.1.0.png, tile.1.1.png, tile.1.2.png, etc., and then so on to tile.10.1.png, tile.10.2.png, etc. And then when you get a directory listing of them, the OS sorts them like this:

tile.1.0.png
tile.1.1.png
tile.1.2.png
...
tile.10.0.png
tile.10.1.png
tile.10.2.png
...
tile.19.0.png
tile.19.1.png
...
tile.2.0.png
tile.2.1.png

So when you go to CONVERT them, and your conversion program expects consecutive tiles, your output ends up garbage because the directory is sorted wrong.

Which goes back to scripts. Heh. Scripts are your Best Buddy. Thanks again guys for reminding me of them!

Anyway, onward!
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 06, 2014, 10:34:18 pm
K. Got the topo's and flipped the y's. Awesome!

That's pretty much got all of the other stuff combined into one set of tiles. Rivers, forests, swamps, lakes, roads. It's definitely nice for pinpointing where towns and cities and places like the entrance to Moria are though!

Although it *is* nice to have things broken out into their own tiles for sure.

I can't thank you enough. You guys are super.
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 08, 2014, 06:51:52 am
Woo hoo! Got it all exported and converted finally.

Found I had mis-converted x7 y2 though, so redoing that.

Must...sleep....
Title: Re: 8 bit Heightmap [for Minecraft]?
Post by: Morcrist on March 08, 2014, 12:09:43 pm
(http://morcrist.com/images/minecraft/bigmap-thumb.jpg) (http://morcrist.com/images/minecraft/bigmap.jpg)

Each pixel is 4x4 blocks in Minecraft. Total area of the map is 40960 x 40960.

EDIT: Feth. I see I missed another piece in the bottom right corner. I'll get it later. Lol.

EDIT: Fixed. Zipped the Minecraft files are 5.6 GB. Unzipped they take up 25 GB.

Now to work!

1) 1st pass topo - COMPLETED

2) 1st pass forests/rivers/lakes

3) 1st pass mountains

4) biomes!

Now for #2....