Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Morcrist]

I'm in the middle of a game project for a studio and all of my me-dem projects are on a backup hd because I ran out of space on my computer. So I can't render anything out. The old renders are nowhere near the finished state of the latest terrain, it's not worth using to be honest. I can have a look to see if I still have the original output files from the last render before it got converted to the Outerra native format. Other than that, folks will have to wait a couple of months- could be a little as a few weeks- before I can go again.
 Our "policy" for want of a better word, is to not give out the terrain at full res. We could give you a 10K terrain and you could always upsample it.

I'm gonna' respond here since I don't want to off-topic the intro thread.

Since Minecraft has a height limit of 256, an 8 bit heightmap would be sufficient for my needs. I have no idea how getting an 8 bit heightmap from the elevation data would work, but if I have to wait...I'll wait.

When you say '10K terrain' what exactly do you mean? What's the dimensions? As a reminder I'm shooting for at least 24000x24000. 48000x48000 would be better!

So...no 8 bit greyscale heightmaps lying around? Doesn't have to be latest version or anything.

Take care.

[Redrobes]

I thought middle earth was done in mine craft. I saw the Hobbiton and Bree on some forum somewhere. You can check out the galleries I am not sure we have an 8 bit version anywhere. All of the stuff we use is float based height. I am not sure that 256 levels would adequately encode a terrain height. If you had a mountain 25600 ft tall then your res would be 100ft per value.

[monks]

We've got a 40K map in 64 tiles I think. I can't guarantee that it's the latest terrain. The folder says "previous terrain" so I think it's the previous terrain we ran- but again I can't guarantee that.  It's generally too high to be realistic, but it is nice visually. You can have those but you'll have to reformat and resize them yourself.
 I might even have the latest one. What's the name of the folder in the Mainmap directory where tiles are output Robes?

monks

[Redrobes]

I dont think we have a greyscale bitmap of the heights only. I think we would need to write a script to generate them if we needed it. We have only got HF2 type. We have a black and white only mask of the land sea at 10K res which is a good start. And we have a black and white mask of the hilly ground at same res and we have a mountain mask as well so we have a two bit solution... Those are in the preprocess 10240 section of the main map files and the Tiles Level_0_10240 section. Not bad for Minecraft tho -maybe

[monks]

Looking at the scripts, the tiles are output in C:\MEDEM\Mainmap\CalcTiles\Level_0_5000

I've got the latest run in there in the OuteraHeight. dir Yep 64 5K hfz. If you want to knock up a script Robes, convert them to 8 bit or whatever he wants them in...or if you're really busy I could just open them in WM and spit them out. That's if my computer has the hd space for the processing. I'm down to my last 150 GB. Prob easier with WM...I'll try it.
What format Morcrist? World Machine has TGA, RAW, BMP....

monks

[Morcrist]

The easiest thing for you guys. Scratch what I said about Global Mapper. My 2 week trial runs out Feb 3, but I just found out I apparently forgot to read about a hard limit on the number of exports I can do. Woopsie!

So yeah, whatever image format is easiest/most space efficient for you. I've got paintshop pro so I can convert to whatever I need. TGA, BMP, JPG even. Any will do.

You mentioned '40K', and '10k' again. What exactly do those mean in terms of resolution? I keep thinking in terms of pixels. Is that right? 40,000 x 40,000 pixels? Or is that kilometers? 40 km? I'm clueless! Heh.

Meanwhile a few days ago I actually opened up NW Middle-Earth in paintshop pro and started a new image. It's about 8000x5000 pixels and represents an area around 1700x1200 miles. Uhh, that'd make it like 1 pixel ~= 360 meters I think since 90 pixels was 20 miles on the map. I plan on upscaling it at least 3x to 24000x15000 and hopefully 48000x30000.

So anyway I immediately started a new layer filled with black and began tracing the coast with a 32 (0=bottom, 255=max height in minecraft). After a few minutes of tracing the coast I stopped, zoomed out...rofl. I had only gotten to like the Sea of Lhun, starting from the top northwest part of the map. Tedious. I eventually finished the coast last night. It's just a very small taste of the work involved in doing something like an entire map from scratch, I'm sure, but I've got a good imagination! Super kudos to you guys once again. /salute

I plan on doing some mountains tonight starting with the peaks and then working my way down. I don't like what I've seen in some other minecraft versions of NW ME where the water is all at the same level. Like globally. I know in real life water doesn't make 1 meter (1 block) drops as it flows to the sea, but I'd like to model the fact that the height above sea level gradually goes up from the coasts towards the mountains. And consequently you've got bodies of water like whatever lake feeds the falls of Rauros at a much higher elevation than say, the Anduin. And rivers would drop a block at a time over their concourse as they head towards sea level.

Just gabbing at this point. I'd love to see what I could do/how much time it could save me in my effort to get something into minecraft by using some of your data, so whenever you're able I'd be much appreciative!

And speaking of data, what did YOU guys use for your source way back? From what I've read online you guys are using the same source as that link above? ICE's MERP module maps? Just curious.

Thanks in advance, and take care.

[monks]

Oh I know about tedious....the contour map I created- in fact the very first map after stitching the ICE maps together- was drawn at 12.5K (12,500 x 12,500 pixels). The contours were drawn with a 1 pixel brush. Look at your computer screen, see 1 pixel? Imagine drawing with that in a sea of white, then stack 12 of your computer monitors on top and to the side of your screen...lol I worked on it literally 14 hours a day for 3 months solid without a break. Honestly don't know what possessed me to do that...it had its own problems. Because you got lost in it, I had to make a first pass with a larger red brush. I'd use this a guide in the 2nd pass. Also, it would be so easy to get gaps of the odd pixel here and there which you had to fix because you needed an unbroken line. If you painted too quickly this would happen so you could only go so fast. So I had to make a 3rd pass over the lines to check them. Not to mention actually working out the contours from the ICE height map (Guide to M-E I think?) and the height refs I compiled from the scenarios. Another thing, you have to make sure that your 1 pixel lines always connect with a face, rather than on a corner, otherwise they get treated like broken lines (can't remember -think it was the raster to vector process). You had to go through thee lines with a 1 pixel and dot the corrections- now that was super tedious. Luckily I had my intuos and set up the buttons to pan/zoom, etc. No way I could have done that with a mouse.
http://www.me-dem.me.uk/galleries/Map_images/ContourswithRivers.png

 That's why you need contours- for the realistic water levels. I don't know of any better way of doing it. But it's a pita, and a bit of an obscure workflow- more GIS than anything else. It's like a table-top effect- like with model railways on a table tennis table. You can see essentially it's all flat with stuff plonked on it.
 For Minecraft though you could possibly get away with using greyscale heights and then digging out the rivers, but I still think contours is a more reliable way to create the rivers properly. The problem is if you use greyscales and then blur them together, blur is a uniform operation, and you don't necessarily want that since you end up closing off narrow river valleys. Contour interpolation on the other hand preserves the exact height at those points.
 There is a better way now as of last year. It took several phDs mind you to pull it off. Funnily enough, this is exactly how I said it needed to be done some years ago, including the polygonal watersheds and Strahler stream ordering. I wish I could program!


Really you need all of your data in a GIS vector format so you can render to planets with scale independence. Unfortunately .ai or .svg don't support per vertex z, so you can't use those. You can convert raster to vector with a little app called WinTopo which I use. You have to tag all of the vector contours with z values- another big and tedious job. At some point you also have to connect all broken discontinuous vector contours as well...because WinTopo algorithm tends to break up lines into smaller segments.
 Then you got to to create all of your rivers and roads.....ouch... .., I'm so glad I'm looking back at it now cos I don't think I'd have the patience or sanity for it anymore.

I'll try outputting 40,000 x 40,000 (40k) in 64 5K tiles in 8 bit bmp format.

monks

[Redrobes]

If your able to do that Monks then that would be useful. The way I would have to do it is to render out all of the 1600 tiles then resample them to 350 pixel tiles then blend them all back into a 10K square PNG. It could be done but for one thing I dont have the latest height tiles tho I think mine are probably good enough. But its a lot of work to generate them.
If we put a lot of effort into it then it would be possible to generate the map and export the height and water tables out to 8 bit with the colour map for minecraft. We could do the volcanic and trees and water bodies as well. Potentially we could make a MeDem in minecraft but I haven't used that app yet. Just seen my young cousins playing around on it thats all. Personally whilst there isa free Outerra demo for MeDem which also fills in the sub 100m level with procedural fractal landscape and sorts out the 3D terrain generation then there is no point in looking around at any other terrain visualization tool.

I dont think Monks covered that last bit tho. We have a final map of 40K x 40K height points and colour pixels. Its generated off of an export of contour maps at a 10K res. We also input some more hand made or script made data at 10K for some parameters like forced vegetation and temperature maps etc so that the terrain generation tool gets the right sort of output terrain for the area being generated. We export for ViewingDale as a 2D set of maps, geomapper set of map layers for a "google maps" type thing that someone does for us and then also Outerra.

Anyways - I hope Monks can export something. I expect that you will discover that most of the terrain will consist of very low value heights and that the blocking will be quite large. But then that's what Minecraft looks like so maybe thats ok.

[monks]

I'll try it here. I need to know though what format he wants first...

monks

[Morcrist]

Any image file format, actually. Whatever is easier? BMP would be fine.

I really appreciate you taking the time to do this for me. Can't wait to see what it looks like!

Thanks, and take care.

[monks]

40K heightmap @ 64 x 5000 px is up. You should have received a mail from GoogleDrive.
 It's bmp and you may have to flip the tiles and/or ordering scheme to get the correct orientation for you app. It all depends on the app you use. We tend to use Perl scripts here.
 Don't forget to credit us. We don't usually give away our data at full res. In this case 8 bit is fine.
 Let us know how you get on with it.

monks

[Morcrist]

OMG!

That is awesome. Monks, I can't thank you enough. I really, really appreciate that. It took me a while to figure out what I was looking at with it divided into 8x8 chunks, but once I did...sweet!

Only thing is, this work has really underscored the age of my hardware. With even 8 Gb of ram I keep running into Out of Memory errors using my (dated) Paintshop Pro 7.04. Heh. Things are slow too. My cpu is only a dual core Pentium (not even an I-xxx). I created a new 40000x5000 image to mosaic a row of your 5k chunks into and only realized PSP maxed out at 32767 width when I got to the last 2 chunks in the row. Derp.

I've been drooling over MSI's 128 Gb motherboard on newegg.com....

But yeah man. Super awesome! I may have to just create the chunks in minecraft using World Painter one chunk at a time and then use MC Edit to stitch the chunks together. Soon as I get something in-game I'll link a pic or something.

You're the greatest. And crediting you guys is the absolute LEAST I can do. Thank you, thank you, thank you!

Take care!

[monks]

...that's great. Yes it's a beast. I did most of this thing with 8 GB RAM though. It was only right at the end (of this run) I upgraded to 32 GB.  So it should be possible. One more reason to use hfz workflow...compressed GIS format we got developed. bmp are not good. Can't you convert to lossless png? Hmm, no, 8 bit. Right.Is there no 8 bit compressed format? Robes will know.
 I guess you'll have the best Middle-Earth terrain in MineCraft now...well in terms of sourced real world terrain. I'll send you the rivers and roads as well if you can use them. I saw painting of rivers by hand. Is there no way to use an image overlay over the terrain so that you can use it as a reference?
 128 GB...? Crikey...filling up that board might be expensive!

I'm going to delete the terrain tiles later this eve, so make sure you have them all.

monks

[Redrobes]

PNG is lossless so for 8 bit should be fine. There is a 16bit PNG but implementation stability varies. Personally I think a PNG or BMP per tile would be fine but I dont think many apps would handle 40Kx40K images. Generally I believe paint shop pro would be ok up to about 16K then it gets problematic. Ideally you need a 64 bit app and run a 64bit version of windows. Even then tho the image file formats are not geared up for very large image sizes. TIFF is probably the best in that regard but TIFF has lots of issues which is why we dont use them. Its a good idea to stick with tiles and page them in as required.

[Morcrist]

Is there no 8 bit compressed format? Robes will know.

Actually, the tiles you outputted were in 24 bit BMP format. They were 72 Mb each (5000x5000x3 bytes). I'm lowering them to 8 bit as I work on them so that's helping (24 Mb vs 72, or 1.5 Gb vs 4.6).

I guess you'll have the best Middle-Earth terrain in MineCraft now...well in terms of sourced real world terrain.

Heh, thanks to you guys' blood, sweat and tears. But I doubt I'll have anything remotely playable for quite some time. I've been playing around with the tile that has the Lonely Mountain and Withered Heath (x=5, y=3), which brings me to...

I'll send you the rivers and roads as well if you can use them.

How does that work, data wise? I mean, the tiles you sent me are 0->255 elevation maps, basically. What would a "river" tile be composed of? Or a "road" tile?

I saw painting of rivers by hand. Is there no way to use an image overlay over the terrain so that you can use it as a reference?

Oh sure. World Painter allows overlays, and I've used them to great effect in my tinkering.

But I'm wondering something at this point. I've imported the x=5, y=3 tile into World Painter as a heightmap. I can see where, for example, the river coming down from the Lonely Mountain flows down into the Long Lake and out the southern end of it. But it almost looks like the riverbed coming from the mountain was eroded, which is fine and natural looking, but the actual lake itself looks gouged out of the terrain at a uniform depth. As if it were done by hand. Likewise the large swamp to the west of the northern end of Long Lake. How did you guys actually do the rivers and lake beds, etc.? I'm thinking that since you weren't concerned with actual voxels you didn't concern yourself with what was UNDER the water. Heh. You just wanted something that would render correctly as a body of water, yes? It didn't matter how deep the lake was, or the swamp, etc. Just my guess.

Anyway, nothing I can't touch up in the BMP before importing.

There also seem to be some other eroded looking "riverbeds" in places that the topo Middle-Earth map doesn't have. I was curious about those as well. For example the long, thin riverbed that seems to originate east/southeast from the Lonely Mountain (almost halfway to the Iron Hills) and then flows south/southeast all the way down to where it finally meets up with the Celduin. Again, I can't find it on the topo and was just curious why it's there?

128 GB...? Crikey...filling up that board might be expensive!

Yar, but I could start with 32 Gb and work my way from there budget permitting. Heh heh. And there's the rub. It's not permitting right now, nor for the forsee-able future.

Thanks again! Take care.