OpenSFS News

Choose Lustre and attend LUG17 at IU

By jfranklin

Regular Registration will be closing May 1 for the Lustre User Group 2017 conference, May 30-June 2, 2017. There is significant savings for registering by this date. You can see the full conference agenda and register at https://opensfs.org/lug-2017/ .

You don’t want to miss this year’s conference, featuring the latest Lustre developments and special events for connecting with your colleagues – like an opening reception sponsored by DDN at IU’s Cyberinfrastructure Building, Dinner and a Movie (Breaking Away) at the historical IU Auditorium sponsored by Intel, and a pub crawl sponsored by HGST/WARP.

Registration will be capped at 200 attendees.  Only a few late registrations will be accepted – and only if capacity has not been reached.

The hotel block for LUG 2017 attendees will expire soon.  A few rooms are still available at the IMU, but be sure to book by the hotel reservation deadline April 25, 2017 at 11:59pm EST.
Questions about the conference? Contact [email protected]

We look forward to hosting you in Bloomington!

About PTI
The Pervasive Technology Institute at Indiana University is a world-class organization dedicated to the development and delivery of innovative information technology to advance research, education, industry and society. Since 2000, PTI has received more than $50 million from the National Science Foundation to advance the nation’s research cyberinfrastructure. Established by a major grant from the Lilly Endowment, the Pervasive Technology Institute brings together researchers and technologists from a range of disciplines and organizations, including the IU School of Informatics and Computing, the IU Maurer School of Law and the College of Arts and Sciences at Bloomington and University Information Technology Services at Indiana University.

3rd International Workshop on the Lustre Ecosystem: Support for Deepening Memory and Storage Hierarchy

By jfranklin

Hanover, Maryland
July 25-26, 2017

The Lustre parallel file system has been widely adopted by the high-performance computing (HPC) systems as an effective mechanism for managing large-scale storage and I/O resources. Lustre is an open-source parallel file system technology and heavily used by the world’s fastest HPC systems. Lustre achieves unprecedented aggregate performance by parallelizing I/O over file system clients and storage targets at extreme scales. Large-scale checkpoint storage and retrieval, which is characterized by bursty I/O from coordinated parallel clients, has been the primary driver of Lustre development over the last decade.

With the introduction of non-volatile storage technologies, many HPC centers are seeing a proliferation of I/O layers in the end-to-end storage hierarchy that place new demands on Lustre. Effectively managing the node-local memory and these new layers is a new challenge for Lustre and requires new technologies and data management policies to be developed to effectively handle data storage and movement across the I/O stack.

In July of 2017, the 3rd International Workshop on the Lustre Ecosystem will be held in Hanover, Maryland. This workshop series is intended to help explore improvements in the performance, flexibility, and usability of Lustre for supporting diverse application workloads and diverse HPC architectures. The past workshops have helped culminate a discussion on the open challenges associated with enhancing Lustre for diverse applications and architectures, the technological advances necessary, and the associated impacts to the Lustre ecosystem. The 3rd International Lustre Ecosystems Workshop will present a series of invited talks from industry, academia, and US National Laboratories focusing on:

  • Lustre Node-Local Memory Management
  • Multilayered Lustre Storage Architectures
  • Data Flow in Lustre across Multiple I/O Stacks
  • Data Management and Handling in Lustre across Multiple I/O Stacks
  • Data Resiliency and Replication Mechanisms in Lustre across Multiple I/O Stacks
  • Data Provenance in Lustre across Multiple I/O Stacks

See the conference information at http://lustre.ornl.gov/ecosystem-2017/index.html

LUG17 abstracts due 2/26 – early bird ends 3/15

By jfranklin

Want to present at LUG? Be sure to submit your abstract by the February 26th deadline: https://easychair.org/cfp/LUG2017. Accepted abstracts will be notified in time to take advantage of early bird registration.

We have a great week planned – including a range of presentations on best practices and boundary-pushing deployments as well as a hackathon.

Networking opportunities abound with an opening reception sponsored by DDN, Dinner and a Show sponsored by Intel, and a pub crawl sponsored by WARP/HGST.

See the conference overview at lug17.sched.com/list/descriptions/.

Time is running out for LUG17 early bird registration! Register now at opensfs.org/lug-2017/ to take advantage of the early bird rate of $499. After March 15, fees increase substantially.

Questions about the conference? Contact [email protected].

We look forward to seeing you in Bloomington!

LUG 2017 – Call for Presentations

By jfranklin

The Lustre User Group (LUG) conference is the industry’s primary venue for discussion and seminars on the Lustre parallel file system.

This year’s conference is being held in Bloomington, Indiana from May 30-June 2nd, 2017. For more information, visit the LUG 2017 web page.
The LUG Planning Committee is particularly seeking presentations on:

  • experiences running the newer community releases – 2.8 and 2.9 – in production
  • experiences using the new features in those releases (DNE2, SSK, UID/GID mapping, etc.)
  • best practices and practical experiences in deploying, monitoring and operating Lustre
  • pushing the boundaries with non-traditional deployments

For more information, or to submit an abstract, please visit the Call for Presentations web page.

Lustre 2.9 Released

By jfranklin

We are pleased to announce that the Lustre 2.9.0 Release has been declared GA and is available for download . You can also grab the source from git

This major release includes new features:

  • UID/GID Mapping  Provides the capability to remap user and group IDs based on node address so as to simplify administration when sharing access between security domains   (LU-3291)
  • Shared Key Crypto Identifies clients/servers and authenticates/encrypts RPCs  (LU-3289)
  • Subdirectory Mounts  Offers the option for clients to be able to mount only a defined subset of the filesystem (LU-29)
  • Server Side Advise and Hinting A Lustre equivalent of fadvise that provides hints to the server about the nature of the data access (willread, dontreed) so that appropriate steps can be taken. (LU-4931)
  • Large Bulk IO Allows the configuration of larger options to be able to get more efficient network/disk IO . (LU-7990)
  • Weak Updates Lustre support for weak updates (KMP) has been added. Please note the associated name changes for the RPMs . (LU-5614)

Fuller details can be found in the 2.9 wiki page (including the change log and test matrix)

The following are known issues in the Lustre 2.9 Release:

  • LU-8313 – Deployments using kerberos that have upgraded from an earlier release will need to manually add a–k flag to svcgssd in order to maintain compatibility
  • LU-8880 – Kerberos does not behave in a consistent manner when deploying in a DNE configuration

These issues are being actively worked and have proposed fixes under testing and review.

Please log any issues found in the issue tracking system 

Thanks to all those who have contributed to the creation of this release.

OpenSFS Reinvents Itself with a Focus on User Needs

By jfranklin

November 11, 2016

BLOOMINGTON, Ind.—Lustre is the file system of choice for the majority of the world’s most powerful supercomputers, helping researchers extract knowledge from data to drive discovery around the world.

Today, the nonprofit organization dedicated to the success of the Lustre file system, Open Scalable File Systems Inc., or OpenSFS, announces organizational changes to give its users—the researchers and scientists who use Lustre every day—more of a say in how OpenSFS moves forward.

One of the first changes affects the OpenSFS board. In late October 2016, a new interim board consisting of users representing academia, business and the US DOE national laboratories was installed and officers were elected.

Members are:

  • Stephen Simms, Indiana University, President
  • Sarp Oral, Oak Ridge National Laboratory, Vice President
  • Rick Wagner, Globus, Secretary
  • Steve Monk, Sandia National Laboratory, Treasurer
  • Shawn Hall, BP, Director-at-Large

This board will remain in place until an election at next year’s Lustre User Group meeting, May 30 to June 2, 2017, in Bloomington, Indiana. More information about that meeting can be found here: https://opensfs.org/lug-2017/.

Now the time has come for those who rely most on Lustre—its users—to guide OpenSFS into the future. Stephen Simms, OpenSFS president and manager, high performance file systems, Indiana University

“OpenSFS has reinvented itself as an organization primarily focused on the needs of Lustre users,” said Stephen Simms, OpenSFS president and manager of high performance file systems at Indiana University. “The ramifications of this change is that the organization will be guided by the needs of its users with support from vendors. We encourage both users and vendors to join and participate.”

Other changes include:

  • A revamped dues structure to encourage wider participation. Members can join for $1,000, and Lustre vendors can join for $5,000
  • Elected leadership
  • A user-run Lustre User Group meeting
  • Opportunities to participate in a requirements gathering exercise connecting our members with vendors

“OpenSFS has accomplished many great things since its inception, providing leadership, labor and capital that have improved Lustre and ensured its place in the HPC ecosystem. Now the time has come for those who rely most on Lustre—its users—to guide OpenSFS into the future,” said Simms.

If you are interested in joining OpenSFS or have questions, please visit: https://opensfs.org/join/ or send an email to [email protected]. OpenSFS will also be attending SC16. Please visit https://opensfs.org/events/ for more information on our event schedule and opportunities to meet with us.

Lustre 2.8.0 Released at LUG 2016 and Declared General Availability

By jfranklin

OpenSFS Funds Additional Security, Support Performance, and Efficiency Development for Lustre, Re-Elects OpenSFS Community Representative Director

BEAVERTON, OR–(Marketwired – Mar 31, 2016) – Open Scalable File Systems (OpenSFS) today announced the release of Lustre® 2.8.0, the fastest and most scalable parallel file system. OpenSFS, founded in 2010 to advance Lustre development, is the premier non-profit organization promoting the use of Lustre and advancing its capabilities through coordinated releases of the Lustre file system.

Large portions of new feature development were funded by OpenSFS through dues collected from participants. Atos also contributed to the new features found in 2.8.0. Lustre now boasts added security and data protection, in addition to the following new features:

  • Distributed Namespace (DNE) Asynchronous Commit of cross-MDT updates for improved performance.
  • Remote rename and remote hard link functionality.
  • LFSCK Phase 4 Performance and efficiency improvements to the online filesystem consistency checker.
  • Red Hat 7.x server support (this release offers support for both servers and clients with RHEL 7.2)
  • Capability to enforce SE Linux security policies for Lustre clients.
  • Multiple Metadata RPCs improves the multi-threaded metadata performance of a single client.

“Lustre is a the most popular storage system for HPC, being on nine of the top 10 systems and over 70 of the top 100 in the top500.org list,” says Brent Gorda, General Manager of Intel’s High Performance Data Division. “The 2.8.0 release adds features that keep with the philosophy of performance and scale while adding features that are attractive to our enterprise and commercial customers. With these features, Lustre will continue to grow as the parallel file system of choice for HPC and enterprise technical computing.”

The Lustre User Group (LUG) conference, now in its 14th year, is the industry’s primary venue for discussion and seminars on the Lustre parallel file system and other open source file system technologies. The second annual Lustre Developer Day will be held in Portland, Oregon, and will be followed by two and a half days of LUG presentations. This year’s LUG features more sponsors than any previous LUG: Cray, DDN, HP Enterprise, Intel, Seagate, WARP Mechanics, Aeon Computing, SGI, and Supermicro.

OpenSFS also announced Steve Simms, manager of high performance file systems at Indiana University, has been re-elected as 2016 OpenSFS Community Representative Director on the OpenSFS Board of Directors. “I am pleased to serve another year on the OpenSFS board,” said Simms. “We continue to see the growth of Lustre acceptance as the features that keep Lustre the fastest parallel file system available continue, while we add features for enterprise and commercial use. I look forward to continuing to help this growth and wider deployment in the commercial arena.”

According to a statement by the OpenSFS Board of Directors, “Steve has been a long-time contributor to OpenSFS, bringing his knowledge, expertise, and vision in driving the growth and acceptance of Lustre and the success of OpenSFS. We are excited to be working with him for another year.”

More information about LUG 2016 can be found at https://opensfs.org/events/lug-2016. Lustre 2.8.0 is available for immediate download at: https://downloads.hpdd.intel.com/public/lustre/lustre-2.8.0.

* All trademarks are the property of their respective owners. OpenSFS is a registered trademark of Open Scalable File Systems, Inc. Lustre is a registered trademark of Seagate Technology PLC in the United States. For more information visit www.opensfs.org.

About Lustre
The Lustre file system is an open source parallel file system widely deployed at major computational centers worldwide, including many of the Top500. Currently seven of the TOP10 supercomputing sites, and over 70% of TOP100 sites, utilize Lustre. OpenSFS plays a pivotal role in the open source Lustre community, through roadmap definition, funding of feature releases, and the OpenSFS Lustre Working Group (LWG), which helps with helps identify requirements for new Lustre features, testing, and coordination of new releases.

About OpenSFS
Open Scalable File Systems, Inc.® is a strong and growing nonprofit organization dedicated to the success of the Lustre® file system. OpenSFS was founded in 2010 to advance Lustre, ensuring it remains vendor-neutral, open, and free. Since its inception, OpenSFS has been responsible for advancing Lustre and delivering new releases on behalf of the open source community. Through working groups, events, and ongoing funding initiatives, OpenSFS harnesses the power of collaborative development to fuel innovation and growth of the Lustre system worldwide.

 #  #  #

CONTACT INFORMATION

Contact
OpenSFS Administration
[email protected]
503-619-0561

Top 5 Reasons To Attend LUG 2016

Top 5 Reasons To Attend LUG 2016

By jfranklin

On April 5-7th, OpenSFS is hosting the 14th annual Lustre® User Group (LUG) conference at the Hilton Portland and Executive Tower in Portland, Oregon! LUG 2016 will bring together industry leaders, end-users, developers, and vendors to talk Lustre, contribute to the community, and move the technology forward.

Here are the top 5 reasons why you should attend LUG 2016:

5 – Free Beer!
LUG wouldn’t be LUG without free beer! One of the highlights of LUG is the opportunity to catch up with old Lustre friends and make new Lustre friends. It’s even better with free beer from local Portland breweries. Join us on Tuesday, April 6th for a networking reception with sweeping views of the Portland skyline. And free beer.

4 – Network with Your Peers and Connect with Key Influencers
Meet and mingle with the people who “speak Lustre.” LUG 2016 is your chance to network, collaborate and share insight with developers, end-users, and other members of the Lustre community over meals, during breaks, and at a special evening reception.

3 – Stay Current on Feature Developments
Dynamic sessions from engineers working on features for upcoming community releases of Lustre.

2 – Visit the City of Roses
The venue for LUG 2016 is the Hilton Portland and Executive Tower, located in the heart of Downtown Portland. Take in all that Portland has to offer, including a thriving food cart scene, over 60 craft breweries in the city limits, and much more!

1 – High Bandwidth, Low Latency Connections to Lustre Developers, Admins, and Users
Prior attendees know that LUG is a firehouse of Lustre information. Get caught up on what’s going on with Lustre, talk with others about Lustre challenges you are facing, understand how others are using Lustre, and what’s coming down the road.

Register by March 11th to take advantage of the special early-bird registration rate!

Lustre Developer Day

Want more Lustre or a chance to talk specifically with Developers? Be sure to arrive a day early for the Lustre Developer Day. Developers looking for a focused opportunity to present their ideas, share current work, and learn from others will want to attend our 2nd annual Lustre Developer Day on April 4th.  Learn More!

If you have any questions, please feel free to contact [email protected].

We look forward to seeing you in Portland at LUG 2016!

LUG 2016: Registration Opens February 1

By jfranklin

Lustre® User Group Conference 2016
April 5-7, 2016
Hilton Portland and Executive Tower
Portland, OR

Are you interested in connecting with thought leaders and developers who are pushing the boundaries with Lustre? Then join us for LUG 2016, the industry’s primary venue for discussion and seminars on the Lustre parallel file system and other open source file system technologies.

  • During this 2.5-day event, attendees will have the opportunity to:
  • Hear from the world’s leading developers, administrators, solution providers, and users of Lustre
  • Be an active participant in industry dialogue on best practices and emerging technologies
  • Explore upcoming developments of the Lustre file system
  • Immerse in the strong Lustre community, working collaboratively to further the development of Lustre

Online registration will open on February 1, and we will be announcing the agenda in the coming weeks. Watch for our early-bird registration to save $50 off your registration rate!

  • Early Bird Registration = $525
  • Regular Registration  = $575

Visit the LUG website for real-time updates on the agenda and other event news.

Lustre Developer Day
We’re excited to host our 2nd annual Lustre Developer Day on Monday, April 4. Developers looking for a focused opportunity to present their ideas, share current work, and learn from others will not want to miss this one-day event! The Developer Day will provide newer developers with an opportunity to listen, learn, and network while providing veteran developers with a forum for socializing current work and issues.

More information about the Developer Day, including agenda and registration, is available on the Developer Day web page.